locked
hierarchical prior - use posteriors of the previous data batch as priorors of new batch RRS feed

  • Question

  • Hi,

    I have a Bayesian network model, and several batches of data.

    In the answer of my previous question here, I understand that if I have a hierarchical prior, I cant just use posteriors of batch N, to update priors of batch N+1 (like what is done in CyclingTime example).

    And I'm now using this update method:

    Variable<Gaussian> wMsg = Variable.Observed(Gaussian.Uniform()).Named("wMsg"); Variable.ConstrainEqualRandom(w, wMsg); w.AddAttribute(QueryTypes.Marginal); w.AddAttribute(QueryTypes.MarginalDividedByPrior); InferenceEngine engine = new InferenceEngine(); Gaussian wMarginal = Gaussian.Uniform(); for (int batch = 0; batch < dataBatches.Length; ++batch) { nItems.ObservedValue = dataBatches[batch].Length; x.ObservedValue = dataBatches[batch]; Console.WriteLine("w after batch {0} = {1}", batch, wMarginal); wMsg.ObservedValue = engine.Infer<Gaussian>(w, QueryTypes.MarginalDividedByPrior);

    }

    Now my question is: I have other variables in the model which are not hierarchical. Should I update their prior distribution just like the ordinary ones, I mean like ( xPrior.observedValue = xPosteriorFromPrevBatch ; )

    Or I should use something like wMsg as an accumulator for all variables and keep this uniformity in the model.

    Thanks  a lot,

    Zahra

    Sunday, August 10, 2014 6:02 PM

Answers

  • Hi Zahra

    There are two slightly different treatments here. (A) When your shared variable is defined directly from a prior, and (B) when your shared variable is defined in terms of another variable. In each case you need to decide whether you want to do (i) full inference or (ii) online inference.

    Ai) For full inference, you loop over the batches many times, but you must take out the effect of when you previously visited a batch otherwise you will double count the data. This mean that after running to convergence on a batch, you must extract and store (keyed by batch index) the 'marginal divided by prior'. Then for any batch, you must set its prior to the current marginal divided by the batch's saved 'marginal divided by prior' (this calculation is equivalent to collapsing all the message from all the other batches in to one prior.

    Aii) For online inference, you only visit batch once, you don't need to save the 'marginal divided by prior', and you just set the prior to the posterior.

    Bi) Collapse the messages as in (a). However, in this collapsed message we must not include the prior message (i.e. we just want the product for 'marginal divided by prior' messages from each other batch), and we implement the constraint enforced by these messages via a ConstrainEqualRandom factor as described in detail in Yordan's reply to your original post.

    Bii) For online inference, you only visit batch once, you just need the 'marginal divided by prior' from the previous batch which you use to set the constraint to the next batch.

    I did not understand your accumulator comment.

    John

    • Marked as answer by RazinR Wednesday, August 13, 2014 10:01 AM
    Monday, August 11, 2014 9:00 AM
    Owner