locked
Compiler limit exceeded for BayesPointMachine with large feature vectors RRS feed

  • Question

  • After being pointed toward using Bayes Point Machines instead of regular Bayesian Networks in another thread, I attempted to use the ImageClassifier project from the Examples directory as a guide. I basically just copied and pasted the BayesPointMachine.cs class into my own project and modified the ItemModel class to suit my needs. When I'm using training datasets with small feature vectors, this model seems to work fine, although the predictions can be a bit off. To increase accuracy, I tried using much larger training datasets with larger feature vectors, which results in some compiler errors, names CS1034 and CS1038:

    error CS1034: Compiler limit exceeded: Line cannot exceed 16777214 characters

    error CS1038: #endregion directive expected.

    After setting "trainEngine.ShowMsl = true;", it seems that every data array is being hard coded in the generated model. I tried setting all the variables to IsReadOnly = false, but it still does the same thing, so I feel like I might be doing this wrong.

    Here's my modified code for the BayesPointMachine constructor:

            public BayesPointMachine(int nFeatures, double noise)
            {
                // Training model
                nTrain = Variable.New<int>().Named("nTrain");
                nTrain.IsReadOnly = false;
    
                Range trainItem = new Range(nTrain).Named("trainItem");
                trainingLabels = Variable.Array<bool>(trainItem).Named("trainingLabels");
                trainingLabels.IsReadOnly = false;
    
                trainingItems = Variable.Array<Vector>(trainItem).Named("trainingItems");
                trainingItems.IsReadOnly = false;
    
                weights = Variable.Random(new VectorGaussian(Vector.Zero(nFeatures),
                    PositiveDefiniteMatrix.Identity(nFeatures))).Named("weights");
                weights.IsReadOnly = false;
    
                trainingLabels[trainItem] = Variable.IsPositive(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(weights, trainingItems[trainItem]), noise));
    
                // Testing model
                nTest = Variable.New<int>().Named("nTest");
                nTest.IsReadOnly = false;
    
                Range testItem = new Range(nTest).Named("testItem");
                testItems = Variable.Array<Vector>(testItem).Named("testItems");
                testItems.IsReadOnly = false;
    
                testLabels = Variable.Array<bool>(testItem).Named("testLabels");
                testLabels.IsReadOnly = false;
    
    
                if (singleModel)
                {
                    testLabels[testItem] = Variable.IsPositive(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(weights, testItems[testItem]), noise));
    
                    testEngine = new InferenceEngine(new ExpectationPropagation());
                    testEngine.NumberOfIterations = 2;
                }
                else
                {
                    weightPosterior = Variable.New<VectorGaussian>().Named("weightPosterior");
                    weightPosterior.IsReadOnly = false;
    
                    Variable<Vector> testWeights = Variable<Vector>.Random(weightPosterior);
                    testWeights.IsReadOnly = false;
    
                    testLabels[testItem] = Variable.IsPositive(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(testWeights, testItems[testItem]), noise));
    
                    trainEngine = new InferenceEngine(new ExpectationPropagation());
                    trainEngine.ShowProgress = false;
                    trainEngine.NumberOfIterations = 5;
                    trainEngine.ShowMsl = true;
                    testEngine = new InferenceEngine(new ExpectationPropagation());
                    testEngine.ShowProgress = false;
                    testEngine.NumberOfIterations = 1;
                }
    
            }

    Would anybody be able to help me with this issue? I can provide the rest of the source code and data files if needed.

    Thanks,

    -Paul




    • Edited by pcherng Monday, August 5, 2013 11:59 PM
    Monday, August 5, 2013 10:59 PM

Answers

All replies

  • IsReadOnly=false is the default so this should work even without those lines.  Can you send the rest of the inference code?  Perhaps something went wrong when setting observations?  Please check that the original ImageClassifier example does not embed the arrays.  Then you might be able to work out what is the difference with your code.
    Tuesday, August 6, 2013 3:38 PM
    Owner
  • The problem might be in the way that we have defined 'weights' in the example.  Try this definition instead:

    var weightPrior = Variable.Observed(new VectorGaussian(Vector.Zero(nFeatures),
    PositiveDefiniteMatrix.Identity(nFeatures))); weights = Variable<Vector>.Random(weightPrior).Named("weights");

    Tuesday, August 6, 2013 3:43 PM
    Owner
  • Indeed, I believe the original ImageClassifier example did embed the arrays. I tried swapping out the weights definition with your suggestion and I did not get the same compiler exceptions, though I did run into an OutOfMemoryException instead:

    An unhandled exception of type 'System.OutOfMemoryException' occurred in Infer.Runtime.dll

    public void Changed_numberOfIterationsDecreased_nTrain_trainingItems_vVectorGaussian0_trainingLabels(int numberOfIterations)
    		{
    			if (this.Changed_numberOfIterationsDecreased_nTrain_trainingItems_vVectorGaussian0_trainingLabels_iterationsDone==numberOfIterations) {
    				return ;
    			}
    			this.weights_marginal_F = ArrayHelper.MakeUniform<VectorGaussian>(this.VVectorGaussian0);
    			PositiveDefiniteMatrix _hoist2 = default(PositiveDefiniteMatrix);
    			_hoist2 = InnerProductOp.AVarianceInit(this.weights_marginal_F);
    			PositiveDefiniteMatrix _hoist3 = default(PositiveDefiniteMatrix);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				_hoist3 = ArrayHelper.CopyStorage<PositiveDefiniteMatrix>(_hoist2);
    				trainItem = this.NTrain-1;
    			}
    			Vector _hoist = default(Vector);
    			_hoist = InnerProductOp.AMeanInit(this.weights_marginal_F);
    			Vector _hoist4 = default(Vector);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				_hoist4 = ArrayHelper.CopyStorage<Vector>(_hoist);
    				trainItem = this.NTrain-1;
    			}
    			// Create array for 'vdouble__0' Forwards messages.
    			this.vdouble__0_F = new DistributionStructArray<Gaussian,double>(this.NTrain);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				this.vdouble__0_F[trainItem] = Gaussian.Uniform();
    			}
    			DistributionStructArray<Gaussian,double> vdouble__1_F = default(DistributionStructArray<Gaussian,double>);
    			// Create array for 'vdouble__1' Forwards messages.
    			vdouble__1_F = new DistributionStructArray<Gaussian,double>(this.NTrain);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				vdouble__1_F[trainItem] = Gaussian.Uniform();
    			}
    			// Create array for 'vdouble__1_marginal' Forwards messages.
    			this.vdouble__1_marginal_F = new DistributionStructArray<Gaussian,double>(this.NTrain);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				this.vdouble__1_marginal_F[trainItem] = Gaussian.Uniform();
    			}
    			// Message from use of 'vdouble__0'
    			DistributionStructArray<Gaussian,double> vdouble__0_use_B = default(DistributionStructArray<Gaussian,double>);
    			// Create array for 'vdouble__0_use' Backwards messages.
    			vdouble__0_use_B = new DistributionStructArray<Gaussian,double>(this.NTrain);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				vdouble__0_use_B[trainItem] = Gaussian.Uniform();
    			}
    			DistributionRefArray<VectorGaussian,Vector> weights_rep_B = default(DistributionRefArray<VectorGaussian,Vector>);
    			// Create array for 'weights_rep' Backwards messages.
    			weights_rep_B = new DistributionRefArray<VectorGaussian,Vector>(this.NTrain);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				weights_rep_B[trainItem] = ArrayHelper.MakeUniform<VectorGaussian>(this.VVectorGaussian0);
    			}
    			// Message to 'weights_marginal' from Variable factor
    			this.weights_marginal_F = VariableVmpOp.MarginalAverageLogarithm<VectorGaussian>(this.weights_use_B, this.VVectorGaussian0, this.weights_marginal_F);
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				_hoist3 = InnerProductOp.AVariance(this.weights_marginal_F, _hoist3);
    				trainItem = this.NTrain-1;
    			}
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				_hoist4 = InnerProductOp.AMean(this.weights_marginal_F, _hoist3, _hoist4);
    				trainItem = this.NTrain-1;
    			}
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				// Message to 'vdouble__0' from InnerProduct factor
    				this.vdouble__0_F[trainItem] = InnerProductOp.InnerProductAverageLogarithm(_hoist4, _hoist3, this.TrainingItems[trainItem]);
    				// Message to 'vdouble__1' from GaussianFromMeanAndVariance factor
    				vdouble__1_F[trainItem] = GaussianFromMeanAndVarianceOp.SampleAverageLogarithm(this.vdouble__0_F[trainItem], 0.25);
    				// Message to 'vdouble__1_marginal' from Variable factor
    				this.vdouble__1_marginal_F[trainItem] = VariableVmpOp.MarginalAverageLogarithm<Gaussian>(this.vdouble__1_use_B[trainItem], vdouble__1_F[trainItem], this.vdouble__1_marginal_F[trainItem]);
    				// Message to 'vdouble__1_use' from IsPositive factor
    				this.vdouble__1_use_B[trainItem] = IsPositiveOp.XAverageLogarithm(this.TrainingLabels[trainItem], this.vdouble__1_marginal_F[trainItem], this.vdouble__1_use_B[trainItem]);
    			}
    			for(int iteration = this.Changed_numberOfIterationsDecreased_nTrain_trainingItems_vVectorGaussian0_trainingLabels_iterationsDone; iteration<numberOfIterations; iteration++) {
    				for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    					_hoist4 = InnerProductOp.AMean(this.weights_marginal_F, _hoist3, _hoist4);
    					trainItem = this.NTrain-1;
    				}
    				for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    					// Message to 'vdouble__0' from InnerProduct factor
    					this.vdouble__0_F[trainItem] = InnerProductOp.InnerProductAverageLogarithm(_hoist4, _hoist3, this.TrainingItems[trainItem]);
    					// Message to 'vdouble__1' from GaussianFromMeanAndVariance factor
    					vdouble__1_F[trainItem] = GaussianFromMeanAndVarianceOp.SampleAverageLogarithm(this.vdouble__0_F[trainItem], 0.25);
    					// Message to 'vdouble__1_marginal' from Variable factor
    					this.vdouble__1_marginal_F[trainItem] = VariableVmpOp.MarginalAverageLogarithm<Gaussian>(this.vdouble__1_use_B[trainItem], vdouble__1_F[trainItem], this.vdouble__1_marginal_F[trainItem]);
    					// Message to 'vdouble__0_use' from GaussianFromMeanAndVariance factor
    					vdouble__0_use_B[trainItem] = GaussianFromMeanAndVarianceOp.MeanAverageLogarithm(this.vdouble__1_marginal_F[trainItem], 0.25);
    					// Message to 'weights_rep' from InnerProduct factor
    					weights_rep_B[trainItem] = InnerProductOp.AAverageLogarithm(vdouble__0_use_B[trainItem], this.TrainingItems[trainItem], weights_rep_B[trainItem]);
    				}
    				// Message to 'weights_use' from Replicate factor
    				this.weights_use_B = ReplicateOp.DefAverageLogarithm<VectorGaussian>(weights_rep_B, this.weights_use_B);
    				// Message to 'weights_marginal' from Variable factor
    				this.weights_marginal_F = VariableVmpOp.MarginalAverageLogarithm<VectorGaussian>(this.weights_use_B, this.VVectorGaussian0, this.weights_marginal_F);
    				for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    					// Message to 'vdouble__1_use' from IsPositive factor
    					this.vdouble__1_use_B[trainItem] = IsPositiveOp.XAverageLogarithm(this.TrainingLabels[trainItem], this.vdouble__1_marginal_F[trainItem], this.vdouble__1_use_B[trainItem]);
    				}
    				for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    					_hoist3 = InnerProductOp.AVariance(this.weights_marginal_F, _hoist3);
    					trainItem = this.NTrain-1;
    				}
    				this.OnProgressChanged(new ProgressChangedEventArgs(iteration));
    			}
    			for(int trainItem = 0; trainItem<this.NTrain; trainItem++) {
    				// Message to 'vdouble__1_marginal' from Variable factor
    				this.vdouble__1_marginal_F[trainItem] = VariableVmpOp.MarginalAverageLogarithm<Gaussian>(this.vdouble__1_use_B[trainItem], vdouble__1_F[trainItem], this.vdouble__1_marginal_F[trainItem]);
    				// Message to 'vdouble__0' from InnerProduct factor
    				this.vdouble__0_F[trainItem] = InnerProductOp.InnerProductAverageLogarithm(_hoist4, _hoist3, this.TrainingItems[trainItem]);
    			}
    			this.Changed_numberOfIterationsDecreased_nTrain_trainingItems_vVectorGaussian0_trainingLabels_iterationsDone = numberOfIterations;
    		}

    I bolded the part of the code where the exception occurs. There are 121 items and the feature vector is 2750 elements long. Is this too data for this model to handle?

    Tuesday, August 6, 2013 5:50 PM
  • That example was written to only handle short feature vectors.  For your dataset you want to use the Sparse Bayes Point Machine.

    • Marked as answer by pcherng Wednesday, August 7, 2013 8:22 PM
    Tuesday, August 6, 2013 8:29 PM
    Owner
  • Great! I basically just adapted the Sparse Bayes Point Machine from the Samples\C#\BayesPointMachine\BayesPointMachine directory and it seems to work pretty well for my purposes. Thanks again!
    Wednesday, August 7, 2013 8:22 PM