locked
Gaussian Conditional Distribution RRS feed

  • Question

  • Hey Guys,

    I'm new to working with Infer.net, and am struggling to perform a task which I believe should be quite easy! Maybe I'm tackling this in the wrong way so I'd value any input from readers of the forum.

    I've completed all of the exercises/tutorials and have not come accross a solution so I thought I would post.

    Essentially I am trying to learn a 2 dimensional Gaussian distribution. i've seen this accomplished in the Gaussian Mixtures example, so I know how to do this. The problem is, I would like to learn said 2d distribution, then condition upon one of the axes. So for example, should i have a gaussian on something like  (Price,Amount), I would like to specify a price, and work out the posterior distribution on the amount.

    I know one solution would be specify a linear combination of variables (learn a multiplicative factor along with an offset), but i don't want to learn a linear relationship between price and amount. I want to learn the 2 dimensional distribution.

    Any ideas how to perfrom this, or where to start? At the moment I'm modelling this linearly as specified above, but want to progress from this.

    All help appreciated!

    Best,

    Doug

    Friday, March 9, 2012 9:52 AM

Answers

  • Hi Doug

    The following code gives an idea of how to do this. It is inefficient as it stands because it will compile (model compiler not C# compiler) each time, so you should put the two models in classes so that the training model, and the model for the conditionals are just compiled once. The code returns the conditional as a Vector Gaussian so you can use this code for more than two dimensions.

    John

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    
    using MicrosoftResearch.Infer;
    using MicrosoftResearch.Infer.Maths;
    using MicrosoftResearch.Infer.Models;
    using MicrosoftResearch.Infer.Distributions;
    
    namespace ConditionalGaussian
    {
    
    	class Program
    	{
    		public static Tuple<VectorGaussian, Wishart> LearnGaussian(Vector[] obs)
    		{
    			int numData = obs.Length;
    			int dim = obs[0].Count;
    			Variable<Vector> mean = Variable.VectorGaussianFromMeanAndPrecision(
    				Vector.Zero(dim),
    				PositiveDefiniteMatrix.IdentityScaledBy(dim,10.0)).Named("mean");
    			Variable<PositiveDefiniteMatrix> prec = Variable.WishartFromShapeAndScale(
    				100.0, PositiveDefiniteMatrix.IdentityScaledBy(dim, 0.01));
    			Range n = new Range(obs.Length).Named("n");
    			VariableArray<Vector> data = Variable.Array<Vector>(n).Named("x");
    			data[n] = Variable.VectorGaussianFromMeanAndPrecision(mean, prec).ForEach(n);
    			data.ObservedValue = obs;
    
    			var engine = new InferenceEngine(new VariationalMessagePassing());
    			var meanPosterior = engine.Infer<VectorGaussian>(mean);
    			var precPosterior = engine.Infer<Wishart>(prec);
    
    			return new Tuple<VectorGaussian, Wishart>(meanPosterior, precPosterior);
    		}
    
    		public static VectorGaussian Conditional(Tuple<VectorGaussian, Wishart> priors, int observedIndex, double observedValue)
    		{
    			Variable<Vector> mean = Variable.Random(priors.Item1);
    			Variable<PositiveDefiniteMatrix> prec = Variable.Random(priors.Item2);
    			Variable<Vector> v = Variable.VectorGaussianFromMeanAndPrecision(mean, prec);
    			// Initialise v to a proper distribution (to avoid improper messages)
    			v.InitialiseTo(new VectorGaussian(priors.Item1.GetMean(), priors.Item2.GetMean()));
    			Variable<double> observedV = Variable.GetItem(v, observedIndex);
    			observedV.ObservedValue = observedValue;
    			var engine = new InferenceEngine(new VariationalMessagePassing());
    			var vPosterior = engine.Infer<VectorGaussian>(v);
    			return vPosterior;
    		}
    		
    		static void Main(string[] args)
    		{
    			Vector[] obs = new Vector[] {
    				Vector.FromArray(1.0, -2.0),
    				Vector.FromArray(1.5, 1.0),
    				Vector.FromArray(2.0, -1.0),
    				Vector.FromArray(0.5, 2.5),
    				Vector.FromArray(0, -1)};
    			var priors = LearnGaussian(obs);
    			Console.WriteLine(priors.Item1);
    			double val = -2.0;
    			double inc = .5;
    			for (int i = 0; i < 9; i++)
    			{
    				Console.WriteLine(Conditional(priors, 0, val));
    				val += inc;
    			}
    		}
    	}
    }

    • Marked as answer by DougMc412 Thursday, March 15, 2012 10:27 AM
    Tuesday, March 13, 2012 9:52 AM
    Owner

All replies

  • Hi Doug

    The following code gives an idea of how to do this. It is inefficient as it stands because it will compile (model compiler not C# compiler) each time, so you should put the two models in classes so that the training model, and the model for the conditionals are just compiled once. The code returns the conditional as a Vector Gaussian so you can use this code for more than two dimensions.

    John

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    
    using MicrosoftResearch.Infer;
    using MicrosoftResearch.Infer.Maths;
    using MicrosoftResearch.Infer.Models;
    using MicrosoftResearch.Infer.Distributions;
    
    namespace ConditionalGaussian
    {
    
    	class Program
    	{
    		public static Tuple<VectorGaussian, Wishart> LearnGaussian(Vector[] obs)
    		{
    			int numData = obs.Length;
    			int dim = obs[0].Count;
    			Variable<Vector> mean = Variable.VectorGaussianFromMeanAndPrecision(
    				Vector.Zero(dim),
    				PositiveDefiniteMatrix.IdentityScaledBy(dim,10.0)).Named("mean");
    			Variable<PositiveDefiniteMatrix> prec = Variable.WishartFromShapeAndScale(
    				100.0, PositiveDefiniteMatrix.IdentityScaledBy(dim, 0.01));
    			Range n = new Range(obs.Length).Named("n");
    			VariableArray<Vector> data = Variable.Array<Vector>(n).Named("x");
    			data[n] = Variable.VectorGaussianFromMeanAndPrecision(mean, prec).ForEach(n);
    			data.ObservedValue = obs;
    
    			var engine = new InferenceEngine(new VariationalMessagePassing());
    			var meanPosterior = engine.Infer<VectorGaussian>(mean);
    			var precPosterior = engine.Infer<Wishart>(prec);
    
    			return new Tuple<VectorGaussian, Wishart>(meanPosterior, precPosterior);
    		}
    
    		public static VectorGaussian Conditional(Tuple<VectorGaussian, Wishart> priors, int observedIndex, double observedValue)
    		{
    			Variable<Vector> mean = Variable.Random(priors.Item1);
    			Variable<PositiveDefiniteMatrix> prec = Variable.Random(priors.Item2);
    			Variable<Vector> v = Variable.VectorGaussianFromMeanAndPrecision(mean, prec);
    			// Initialise v to a proper distribution (to avoid improper messages)
    			v.InitialiseTo(new VectorGaussian(priors.Item1.GetMean(), priors.Item2.GetMean()));
    			Variable<double> observedV = Variable.GetItem(v, observedIndex);
    			observedV.ObservedValue = observedValue;
    			var engine = new InferenceEngine(new VariationalMessagePassing());
    			var vPosterior = engine.Infer<VectorGaussian>(v);
    			return vPosterior;
    		}
    		
    		static void Main(string[] args)
    		{
    			Vector[] obs = new Vector[] {
    				Vector.FromArray(1.0, -2.0),
    				Vector.FromArray(1.5, 1.0),
    				Vector.FromArray(2.0, -1.0),
    				Vector.FromArray(0.5, 2.5),
    				Vector.FromArray(0, -1)};
    			var priors = LearnGaussian(obs);
    			Console.WriteLine(priors.Item1);
    			double val = -2.0;
    			double inc = .5;
    			for (int i = 0; i < 9; i++)
    			{
    				Console.WriteLine(Conditional(priors, 0, val));
    				val += inc;
    			}
    		}
    	}
    }

    • Marked as answer by DougMc412 Thursday, March 15, 2012 10:27 AM
    Tuesday, March 13, 2012 9:52 AM
    Owner
  • Hi John,

    Great, many thanks for your comprehensive reply! FYI I tried several approaches, but the missing key for me was:

    Variable<double> observedV = Variable.GetItem(v, observedIndex);
                observedV.ObservedValue = observedValue;
                var engine = new InferenceEngine(new VariationalMessagePassing());

    I just didn't realise that you pull Variables from Variable<Vector> types. :-)

    Thanks and best wishes,

    Doug

    Thursday, March 15, 2012 10:26 AM