locked
re 'Expectation propagation' engine in Infer.Net RRS feed

  • Question

  • Dear whom it may concern, 

    I'd like to ask a short question on expectation propagation (EP) engine in Infer.Net. 

    When Infer.Net conducts approximate inference using EP engine, does it assume 'some approximation' on the non-analytic integrals arising during the moment-matching process or else rather provide exact solutions to such integrals?

    If approximation is required, would it be possible to share what method of approximation is used?

    If no approximation is required, would you be able to share how you would cope with multi-dimensional non-analytic integrals (for example, for a simple linear regression model with p number of predictors, you might need to handle p-dimensional integrals with no closed-form solutions.)

    If this is not the right place to ask this type of question, please excuse me and disregard this question!

    Best Regards

    Andy

    Thursday, August 6, 2015 6:15 AM

All replies

  • Non-analytic integrals are usually handled with quadrature.  Multi-dimensional integrals can usually be reduced to low-dimensional integrals.  In the linear regression case, everything is analytic.  For Poisson regression with exponential link function, only the exponential factor produces non-analytic integrals, and those are one-dimensional.  For more details on how specific factors are approximated, see the source code provided in the Infer.NET release.
    Thursday, August 6, 2015 6:42 AM
    Owner
  • Thank you so much for the reply, Tom. 

    I'll surely look into the source code in the Infer.NET release. 

    I'm just wondering if you could elaborate little more on how multi-dimensional integrals can be reduced to low-dimensional integrals. 

    With many thanks,
    Andy
    Thursday, August 6, 2015 8:03 AM
  • See "Expectation propagation for exponential families" where the reduction of dimension is called the "locality property of EP".
    Thursday, August 6, 2015 8:27 AM
    Owner