I am continuing to build a simple Thurstonian model for learning to rank. The scores of two items to rank are Gaussian distributed. The means of these VectorGaussian RVs are Inner Product between a "w" parameters vector (inferred in training; fixed
in prediction) and a respective features vector (given and fixed). The ranking stems from checking three cases when comparing two scores (draw, left > right, right < left) for two respective items in an example. An example is composed of a variable number
of items. The dataset consists of a number of examples.
The training appears to be working as I can infer (1) the posterior for "w" which is a VectorGaussian and (2) a global, shared mean for all draw RVs (Gaussian; one per training example).
The model is written as follows:
using (Variable.ForEach(example))
{
using (Variable.ForEach(item))
{
var mean = Variable.InnerProduct(w, features[example][item]).Named("mean");
scores[example][item] = Variable.GaussianFromMeanAndVariance(mean, 1.0);
}
using (ForEachBlock pairBlock = Variable.ForEach(rank))
{
var idx = pairBlock.Index;
var left = scores[example][idx];
var right = scores[example][idx + 1];
using (Variable.If(Variable.IsBetween(left - right, -drawMargin[example], drawMargin[example])))
ranks[example][rank] = 0;
using (Variable.If(left > right + drawMargin[example]))
ranks[example][rank] = 1;
using (Variable.If(left < right + drawMargin[example]))
ranks[example][rank] = 2;
}
}
The relevant model variables are defined as:
numExamples = Variable.New<int>();
example = new Range(numExamples).Named("exampleRange");
//
// Jagged arrays for (items, features)
//
exampleSize = Variable.Array<int>(example).Named("itemSizes");
item = new Range(exampleSize[example]).Named("itemRange");
scores = Variable.Array(Variable.Array<double>(item), example).Named("scores");
features = Variable.Array(Variable.Array<Vector>(item), example).Named("features");
//
// Jagged array for item pair ranks
//
rankSize = Variable.Array<int>(example);
rank = new Range(rankSize[example]).Named("rankRange");
ranks = Variable.Array(Variable.Array<int>(rank), example).Named("pairwiseRanks");
//
// Draw
//
drawMeanPrior = Variable.New<Gaussian>();
drawMean = Variable.Random<double, Gaussian>(drawMeanPrior);
var drawMargin = Variable.Array<double>(example);
using (Variable.ForEach(example))
{
drawMargin[example] = Variable.GaussianFromMeanAndVariance(drawMean, 1.0).Named("drawMargin");
Variable.ConstrainPositive(drawMargin[example]);
}
//
// Model parameters
//
wPrior = Variable.New<VectorGaussian>();
w = Variable.Random<Vector, VectorGaussian>(wPrior).Named("w");
At prediction stage, I observe all the variables pertaining array sizes, initial priors and feature vectors as in training. The difference this time is that (1) I observe the prior for "w" (it is trained and fixed), (2) observe draw mean prior
and (3) don't observe "rank" as I want to infer it.
To infer pairwise rankings, I provide one single example that is composed of 6 feature vectors (6 items to rank). When I run the inference, I get the error:
'ranks' is not defined in all cases. It is only defined for (vbool7=true)(vbool8=true)(vbool9=true) in:
Intuitively I get that dependent on my pairwise rankings, not all "ranks" elements will be set. I am not sure how to fix this error though.
The full code is on GitHub:
https://github.com/usptact/Infer.NET-LTR/tree/master/Infer.NET-LTR
The data are in SVM-Light format; I used LETOR MQ2007 and MQ2008 datasets for experiments and validation.
