locked
On-line learning on DifficultyAbility RRS feed

  • Question

  • I'm trying to implement on-line learning on the DifficultyAbility estimate. So far I have changed the example to iterate over observations that are (subject, question, response) tuples and I have modified the code to allow on-line learning in batches (with accumulators and whatnot) 

    Currently I am getting poor performance (it looks like the parameters are just not getting any kind of updates). 

    The output:

    27% TrueAnswers correct
    difficulty[0] = Gaussian(0, 1) (sampled from 2.4)
    difficulty[1] = Gaussian(0, 1) (sampled from -0.24)
    difficulty[2] = Gaussian(0, 1) (sampled from -0.21)
    difficulty[3] = Gaussian(0, 1) (sampled from 0.26)
    discrimination[0] = Gamma(1, 1)[mean=1] (sampled from 0.68)
    discrimination[1] = Gamma(1, 1)[mean=1] (sampled from 1.2)
    discrimination[2] = Gamma(1, 1)[mean=1] (sampled from 1.7)
    discrimination[3] = Gamma(1, 1)[mean=1] (sampled from 0.07)
    ability[0] = Gaussian(0, 1) (sampled from 0.58)
    ability[1] = Gaussian(0, 1) (sampled from 0.81)
    ability[2] = Gaussian(0, 1) (sampled from 1.5)
    ability[3] = Gaussian(0, 1) (sampled from 0.33)


    Here is my code:

    static void DifficultyAbility2() { InferenceEngine engine = new InferenceEngine(); Rand.Restart(0); int nQuestions = 100; int nSubjects = 40; int nChoices = 4; Gaussian abilityPrior = new Gaussian(0.0, 1.0); Gaussian difficultyPrior = new Gaussian(0.0, 1.0); Gamma discriminationPrior = Gamma.FromMeanAndVariance(1.0, 1.0); double[] trueAbility, trueDifficulty, trueDiscrimination; int[] trueTrueAnswer; int[][] data = Sample2(nSubjects, nQuestions, nChoices, abilityPrior, difficultyPrior, discriminationPrior, out trueAbility, out trueDifficulty, out trueDiscrimination, out trueTrueAnswer); Range question = new Range(nQuestions).Named("question"); Range subject = new Range(nSubjects).Named("subject"); Range choice = new Range(nChoices).Named("choice"); var ability = Variable.Array<double>(subject).Named("ability"); ability[subject] = Variable.Random(abilityPrior).ForEach(subject); var difficulty = Variable.Array<double>(question).Named("difficulty"); difficulty[question] = Variable.Random(difficultyPrior).ForEach(question); var discrimination = Variable.Array<double>(question).Named("discrimination"); discrimination[question] = Variable.Random(discriminationPrior).ForEach(question); var trueAnswer = Variable.Array<int>(question).Named("trueAnswer"); trueAnswer[question] = Variable.DiscreteUniform(choice).ForEach(question); ability[subject].InitialiseTo(Gaussian.FromMeanAndVariance(0.0, 100.0)); difficulty[question].InitialiseTo(Gaussian.FromMeanAndVariance(0.0, 100.0)); discrimination[question].InitialiseTo(Gamma.FromMeanAndVariance(1.0, 1.0)); var difficultyMessage = Variable.Observed(Util.ArrayInit(nQuestions, q => Gaussian.Uniform()), question); var discriminationMessage = Variable.Observed(Util.ArrayInit(nQuestions, q => Gamma.Uniform()), question); var abilityMessage = Variable.Observed(Util.ArrayInit(nSubjects, q => Gaussian.Uniform()), subject); using (Variable.ForEach(question)) { Variable.ConstrainEqualRandom(discrimination[question], discriminationMessage[question]); Variable.ConstrainEqualRandom(difficulty[question], difficultyMessage[question]); } using (Variable.ForEach(subject)) { Variable.ConstrainEqualRandom(ability[subject], abilityMessage[subject]); } discrimination.AddAttribute(QueryTypes.Marginal); discrimination.AddAttribute(QueryTypes.MarginalDividedByPrior); difficulty.AddAttribute(QueryTypes.Marginal); difficulty.AddAttribute(QueryTypes.MarginalDividedByPrior); ability.AddAttribute(QueryTypes.Marginal); ability.AddAttribute(QueryTypes.MarginalDividedByPrior); Variable<int> nObservations = Variable.New<int>().Named("nObservations"); Range obs = new Range(nObservations).Named("observation"); var subjectOfObs = Variable.Array<int>(obs).Named("subject of obs"); var questionOfObs = Variable.Array<int>(obs).Named("question of obs"); var response = Variable.Array<int>(obs).Named("responses"); using (Variable.ForEach(obs)) { var q = questionOfObs[obs]; var advantage = (ability[subjectOfObs[obs]] - difficulty[q]); var advantageNoisy = Variable.GaussianFromMeanAndPrecision(advantage, discrimination[q]); var correct = (advantageNoisy > 0.0); using (Variable.If(correct)) response[obs] = trueAnswer[q]; using (Variable.IfNot(correct)) response[obs] = Variable.DiscreteUniform(nChoices); } engine.SaveFactorGraphToFolder = "Tutorial"; engine.NumberOfIterations = 10; engine.ShowProgress = false; int nObsObs = data.Length; subject.AddAttribute(new Sequential()); // needed to get stable convergence question.AddAttribute(new Sequential()); // needed to get stable convergence

    //online learning in mini-batches int batchSize = 10; IList<Gaussian> difficultyMarginal = Util.ArrayInit(nQuestions, q => Gaussian.Uniform()); IList<Gamma> discriminationMarginal = Util.ArrayInit(nQuestions, q => Gamma.Uniform()); IList<Gaussian> abilityMarginal = Util.ArrayInit(nSubjects, q => Gaussian.Uniform()); int[][][] dataBatches = new int[data.Length / batchSize][][]; for (int batch = 0; batch < dataBatches.Length; batch++) { dataBatches[batch] = data.Skip(batch * batchSize).Take(batchSize).ToArray(); } Console.WriteLine("total iterations: " + data.Length); for (int batch = 0; batch < data.Length / batchSize; batch++) { Console.WriteLine("batch: " + batch + " iteration " + batch * batchSize); nObsObs = batchSize; nObservations.ObservedValue = nObsObs; subjectOfObs.ObservedValue = Util.ArrayInit(nObsObs, r => dataBatches[batch][r][1]); questionOfObs.ObservedValue = Util.ArrayInit(nObsObs, r => dataBatches[batch][r][2]); response.ObservedValue = Util.ArrayInit(nObsObs, r => dataBatches[batch][r][3]); difficultyMarginal = engine.Infer<IList<Gaussian>>(difficulty); discriminationMarginal = engine.Infer<IList<Gamma>>(discrimination); abilityMarginal = engine.Infer<IList<Gaussian>>(ability); difficultyMessage.ObservedValue = engine.Infer<Gaussian[]>(difficulty, QueryTypes.MarginalDividedByPrior); discriminationMessage.ObservedValue = engine.Infer<Gamma[]>(discrimination, QueryTypes.MarginalDividedByPrior); abilityMessage.ObservedValue = engine.Infer<Gaussian[]>(ability, QueryTypes.MarginalDividedByPrior); } //analyze results var trueAnswerPosterior = engine.Infer<IList<Discrete>>(trueAnswer); var difficultyPosterior = engine.Infer<IList<Gaussian>>(difficulty); var discriminationPosterior = engine.Infer<IList<Gamma>>(discrimination); var abilityPosterior = engine.Infer<IList<Gaussian>>(ability); bool doMajorityVoting = false; // set this to 'true' to do majority voting if (doMajorityVoting) { ability.ObservedValue = Util.ArrayInit(nSubjects, i => 0.0); difficulty.ObservedValue = Util.ArrayInit(nQuestions, i => 0.0); discrimination.ObservedValue = Util.ArrayInit(nQuestions, i => 1.0); } int numCorrect = 0; for (int q = 0; q < nQuestions; q++) { int bestGuess = trueAnswerPosterior[q].GetMode(); if (bestGuess == trueTrueAnswer[q]) numCorrect++; } double pctCorrect = 100.0 * numCorrect / nQuestions; Console.WriteLine("{0}% TrueAnswers correct", pctCorrect.ToString("f0")); for (int q = 0; q < Math.Min(nQuestions, 4); q++) { Console.WriteLine("difficulty[{0}] = {1} (sampled from {2})", q, difficultyPosterior[q], trueDifficulty[q].ToString("g2")); } for (int q = 0; q < Math.Min(nQuestions, 4); q++) { Console.WriteLine("discrimination[{0}] = {1} (sampled from {2})", q, discriminationPosterior[q], trueDiscrimination[q].ToString("g2")); } for (int s = 0; s < Math.Min(nSubjects, 4); s++) { Console.WriteLine("ability[{0}] = {1} (sampled from {2})", s, abilityPosterior[s], trueAbility[s].ToString("g2")); } } public static int[][] Sample2(int nSubjects, int nQuestions, int nChoices, Gaussian abilityPrior, Gaussian difficultyPrior, Gamma discriminationPrior, out double[] ability, out double[] difficulty, out double[] discrimination, out int[] trueAnswer) { ability = Util.ArrayInit(nSubjects, s => abilityPrior.Sample()); difficulty = Util.ArrayInit(nQuestions, q => difficultyPrior.Sample()); discrimination = Util.ArrayInit(nQuestions, q => discriminationPrior.Sample()); trueAnswer = Util.ArrayInit(nQuestions, q => Rand.Int(nChoices)); int[] nQuestionsForSubject = new int[nSubjects]; int[][] questionsForSubject = new int[nSubjects][]; for (int s = 0; s < nSubjects; s++) { int[] questions = Rand.Perm(nQuestions); nQuestionsForSubject[s] = Rand.Int(9,nQuestions) + 1; questionsForSubject[s] = Util.ArrayInit(nQuestionsForSubject[s], q => questions[q]); } int totResponses = nQuestionsForSubject.Sum(); int[][] response = new int[totResponses][]; int si = 0; for (int r = 0; r < totResponses; r++) { if (r == nQuestionsForSubject.Take(si + 1).Sum()) si++; response[r] = new int[4]; int qi = questionsForSubject[si][r - nQuestionsForSubject.Take(si).Sum()]; double advantage = ability[si] - difficulty[qi]; double noise = Gaussian.Sample(0, discrimination[qi]); bool correct = (advantage > noise); if (correct) response[r] = new int[] { r, si, qi, trueAnswer[qi] }; else response[r] = new int[] { r, si, qi, Rand.Int(nChoices) }; } return response; }


    Monday, October 2, 2017 3:30 PM

Answers

  • You forgot to propagate the posterior for trueAnswer.  Also, when you are computing abilityMessage, you have already changed difficultyMessage, which will corrupt the results. See the code in this post.
    Monday, October 2, 2017 4:58 PM
    Owner

All replies

  • I tried increasing the number of subjects (40 -> 400) and I noticed some of the posteriors were different (but not all). I also tried initializing the trueAnswer variable

                Discrete[] cinit = new Discrete[nQuestions];
                for (int i = 0; i < cinit.Length; i++)
                    cinit[i] = Discrete.PointMass(trueTrueAnswer[i], nChoices);
                trueAnswer.InitialiseTo(Distribution<int>.Array(cinit));

    but that didn't seem to change anything. 

    Monday, October 2, 2017 4:32 PM
  • You forgot to propagate the posterior for trueAnswer.  Also, when you are computing abilityMessage, you have already changed difficultyMessage, which will corrupt the results. See the code in this post.
    Monday, October 2, 2017 4:58 PM
    Owner
  • Good catch Tom. Can you explain a bit more about abilityMessage and difficultyMessage? What do you mean by corrupt the results? 
    Monday, October 2, 2017 5:06 PM
  • See [Contd] Implementing Online Bernoulli Mixture Model                                
    Monday, October 2, 2017 5:14 PM
    Owner
  • is the appropriate solution to set a temp variable equal to the accumulators before assigning Observed values?

                    var temp1 = engine.Infer<Gaussian[]>(difficulty, QueryTypes.MarginalDividedByPrior);
                    var temp2 = engine.Infer<Gamma[]>(discrimination, QueryTypes.MarginalDividedByPrior);
                    var temp3 = engine.Infer<Gaussian[]>(ability, QueryTypes.MarginalDividedByPrior);
                    var temp4 = engine.Infer<Discrete[]>(trueAnswer, QueryTypes.MarginalDividedByPrior);
    
                    difficultyMessage.ObservedValue = temp1;
                    discriminationMessage.ObservedValue = temp2;
                    abilityMessage.ObservedValue = temp3;
                    trueAnswerMessage.ObservedValue = temp4;

    Monday, October 2, 2017 8:23 PM
  • Yes.
    Wednesday, October 4, 2017 4:40 PM
    Owner