locked
How to do Gaussian multiplication RRS feed

  • Question

  • Dear Infer.net,

    I was looking at the BPM model structure (http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Infer.NET%20Learners%20-%20Bayes%20Point%20Machine%20classifiers%20-%20Modelling.aspx) and noticed that it involves multiplying two values together (features and weights). I've previously tried incorporating such multiplications into my models and have been warned that the factor which handles this operation is classified as 'Experimental' and not well developed and as such I've had trouble getting these models to converge without crashing. Does does the BPM model use this factor and if so does it use any tricks/techniques to ensure correct model convergence?

    Thanks,

    Mark

    Friday, May 1, 2015 10:21 AM

Answers

  • Hi Mark,

    We have to distinguish between multiplying the Gaussians when one of them is observed and when they're both stochastic.

    The BPM's case is the former, because the features are observed (whiled the weights are learned). Here, computing the product shouldn't be a problem and the model should converge well (unless there aren't any correlated features, which slows down convergence).

    Matchbox's case is the latter, because both the user and the item traits are learned. This product is much more difficult due to the multi-modality of the marginal. We prefer to use VMP here, because it can focus on one of the modes. You can however incorporate this in an EP model using the Product_SHG09 factor:

    traitProducts[trait] = Variable<double>.Factor(Factor.Product_SHG09, userTraits[trait], itemTraits[trait]);


    -Y-

    • Marked as answer by MarkG87 Sunday, May 3, 2015 8:13 AM
    Friday, May 1, 2015 2:22 PM
  • "X * Y" will call the EP product factor in an EP algorithm, unless you don't do engine.Compiler.GivePriorityTo(typeof(GaussianProductOp_SHG09)). In such cases I'd always prefer to call the factor explicitly in order to avoid this confusion.

    Matchbox (with features) is a good example of this, because it uses both factors - the EP one (when computing the feature weights, where one of the inputs is deterministic) and the VMP-like one (when computing the traits, where both inputs are stochastic). If you used GivePriorityTo you wouldn't be able to have two different product factors. But running the model in EP and using "*" for the features and Product_SHG09 for the traits allows you achieve the desired result.

    -Y-

    • Marked as answer by MarkG87 Saturday, May 9, 2015 2:40 PM
    Friday, May 8, 2015 1:38 PM

All replies

  • Hi Mark,

    We have to distinguish between multiplying the Gaussians when one of them is observed and when they're both stochastic.

    The BPM's case is the former, because the features are observed (whiled the weights are learned). Here, computing the product shouldn't be a problem and the model should converge well (unless there aren't any correlated features, which slows down convergence).

    Matchbox's case is the latter, because both the user and the item traits are learned. This product is much more difficult due to the multi-modality of the marginal. We prefer to use VMP here, because it can focus on one of the modes. You can however incorporate this in an EP model using the Product_SHG09 factor:

    traitProducts[trait] = Variable<double>.Factor(Factor.Product_SHG09, userTraits[trait], itemTraits[trait]);


    -Y-

    • Marked as answer by MarkG87 Sunday, May 3, 2015 8:13 AM
    Friday, May 1, 2015 2:22 PM
  • Thanks Yordan, that makes a lot of sense. If I wanted to use an EP model would you recommend calling the Product_SHG09 factor directly (as in your example) vs using the 'x * y' override? Is there a difference?
    Wednesday, May 6, 2015 11:48 AM
  • "X * Y" will call the EP product factor in an EP algorithm, unless you don't do engine.Compiler.GivePriorityTo(typeof(GaussianProductOp_SHG09)). In such cases I'd always prefer to call the factor explicitly in order to avoid this confusion.

    Matchbox (with features) is a good example of this, because it uses both factors - the EP one (when computing the feature weights, where one of the inputs is deterministic) and the VMP-like one (when computing the traits, where both inputs are stochastic). If you used GivePriorityTo you wouldn't be able to have two different product factors. But running the model in EP and using "*" for the features and Product_SHG09 for the traits allows you achieve the desired result.

    -Y-

    • Marked as answer by MarkG87 Saturday, May 9, 2015 2:40 PM
    Friday, May 8, 2015 1:38 PM