Dear Infer.NET community:
As I am still learning the details of Infer.NET, I came across an implementation need that I cant' seem to get right after 2 days of trying a couple of things, perhaps I am going in the wrong direction.
I am trying to implement Supervised LDA. The model itself doesn't matter as much as this issue that I am having: Essentially the issue is using the empirical topic counts in the dot product with a gaussian-distributed weight vector, inside a logistic factor
(BernoulliFromLogOdds). I have tried a few things, but can't quite get it. First, I tried using a Multinomial VariableArray instead of a Discrete variable for drawing topics. The issue I ran into was converting the array into a vector type to be used as an
argument to InnerProduct factor. Besides the SetTo, or ConstraintEqual (which gave me problems), I couldn't see any other way. Here is sketch of what I tried to write, as it illustrates the problem (in this case I explicitly assume only two topics and do it
in a rolled out way for illustration purpose):
class Program
{
static void Main(string[] args)
{
Range n = new Range(10); // number of words
Range k = new Range(2); // number of latent topics
// indicates whether nth word is in topic 1
VariableArray<double> indicatorsTopicOne = Variable.Array<double>(n);
// indicates whether nth word is in topic 0
VariableArray<double> indicatorsTopicZero = Variable.Array<double>(n);
// topic draws for each word
VariableArray<int> topics = Variable.Array<int>(n);
// prior on topics
Variable<Vector> theta = Variable<Vector>.DirichletSymmetric(k.SizeAsInt, 1.0);
// prior on topic counts. Probably not a good choice, but the line below it won't work
Variable<Vector> topicCounts = Variable<Vector>.DirichletSymmetric(k.SizeAsInt, 1.0);
topics[n] = Variable.Discrete(theta).ForEach(n);
//Variable<Vector> topicCounts = new Variable<Vector>(); <--- this doesn't work
using(Variable.ForEach(n))
{
Variable<bool> isTopicOne = topics[n] == 1;
using(Variable.If(isTopicOne))
{
Variable.ConstrainEqual(indicatorsTopicOne[n], 1.0);
Variable.ConstrainEqual(indicatorsTopicZero[n], 0.0);
}
using(Variable.IfNot(isTopicOne))
{
Variable.ConstrainEqual(indicatorsTopicZero[n], 1.0);
Variable.ConstrainEqual(indicatorsTopicOne[n], 0.0);
}
}
Variable<double> topicOneCount = Variable.Sum(indicatorsTopicOne);
Variable<double> topicZeroCount = Variable.Sum(indicatorsTopicZero);
topicCounts.SetValueRange(k);
// I tried this below as well (in comments)
//Variable.ConstrainEqual<double>(Variable.GetItem(topicCounts, 0), topicZeroCount);
//Variable.ConstrainEqual<double>(Variable.GetItem(topicCounts, 1), topicOneCount);
// Not sure if setting like this is the right way.
Variable.GetItem(topicCounts, 0).SetTo(topicZeroCount);
Variable.GetItem(topicCounts, 1).SetTo(topicOneCount);
Vector mean = Vector.FromArray(new double[] {1.0,2.0});
PositiveDefiniteMatrix prec = PositiveDefiniteMatrix.Identity(2);
Variable<Vector> w = Variable<Vector>.VectorGaussianFromMeanAndPrecision(mean, prec);
Variable<bool> y = Variable.BernoulliFromLogOdds(Variable.InnerProduct(w, topicCounts));
Variable.ConstrainTrue(y);
InferenceEngine engine = new InferenceEngine(new VariationalMessagePassing());
engine.Infer(w);
}
}
Is there a way to encode what I want to do in Infer.NET, or is it not possible at the moment? I am still struggling a bit in figuring out the right way to encode the model.
Thanks so much for your time and help!
