locked
Inference not supported with EP due to MMath.Logistic(double logistic, double x) RRS feed

  • Question

  • I have a binary logistic regression model as follows.

    	public static void InferCoefficients(Vector[] xObs, bool[] yObs, 
                                           out VectorGaussian wPost, out Gaussian biasPost){
      
    			Variable<int> dataCount = Variable.Observed(xObs.Length).Named("dataCount");
    			int D = xObs[0].Count;
    			Range n = new Range(dataCount).Named("n");
    
    			Vector meanW = Vector.Zero(D);
    			Variable<Vector> w = Variable.VectorGaussianFromMeanAndPrecision(
    				meanW, PositiveDefiniteMatrix.Identity(D)).Named("W");
    
    			// bias term is normally distributed.
    			Variable<double> bias = Variable.GaussianFromMeanAndPrecision(0, 1)
    				.Named("bias");
    
    			VariableArray<Vector> x = Variable.Observed<Vector>(xObs, n).Named("X");
    			VariableArray<double> logisticArgs = Variable.Array<double>(n).Named("logist_arg");
    			logisticArgs[n] = Variable.InnerProduct(x[n], w).Named("inner") + bias ;
    
    			VariableArray<bool> yData = Variable.Observed<bool>(yObs, n).Named("Y");
    			VariableArray<double> probs = Variable.Array<double>(n).Named("probs");
    			probs[n] = Variable.Logistic(logisticArgs[n]).Named("logistic");
    			yData[n] = Variable.Bernoulli(probs[n]);
    			// set observed values
    			x.ObservedValue = xObs;
    			yData.ObservedValue = yObs;

    When I run my code with ExpectationPropagation algorithm, I got the following errors. The factor manager seems to report that EP works on a Logistic factor. May I know what I have to do to solve the problem ? Thanks.

    Unhandled Exception:
    MicrosoftResearch.Infer.CompilationFailedException: MessageTransform failed with 2 error(s) and 0 warning(s):
    Error 0: This model is not supported with ExpectationPropagation due to MMath.Logistic(double logistic, double x). Try using a different algorithm or expressing the model differently in
    MMath.Logistic(logist_arg_use[n])

    .....

    at MicrosoftResearch.Infer.ModelCompiler.GetTransformedDeclaration (ITypeDeclaration itd, System.Reflection.MethodBase method, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes) [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.ModelCompiler.CompileWithoutParams (ITypeDeclaration itd, System.Reflection.MethodBase method, MicrosoftResearch.Transforms.AttributeRegistry`2 inputAttributes) [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.InferenceEngine.Compile () [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.InferenceEngine.BuildAndCompile (Boolean inferOnlySpecifiedVars, IEnumerable`1 vars) [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.InferenceEngine.GetCompiledInferenceAlgorithm (Boolean inferOnlySpecifiedVars, IVariable var) [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.InferenceEngine.InferAll (Boolean inferOnlySpecifiedVars, IVariable var) [0x00000] in <filename unknown>:0
      at MicrosoftResearch.Infer.InferenceEngine.Infer[VectorGaussian] (IVariable var) [0x00000] in <filename unknown>:0



    • Edited by wittawatj Wednesday, January 21, 2015 8:59 PM
    Wednesday, January 21, 2015 8:49 PM

Answers

  • I just tried your code with version 2.6 and it works fine.
    • Marked as answer by wittawatj Thursday, January 29, 2015 1:02 PM
    Thursday, January 22, 2015 1:44 PM
    Owner

All replies

  • InferenceEngine ie = new InferenceEngine();
    
    ie.Compiler.GivePriorityTo(typeof(MicrosoftResearch.Infer.Factors.LogisticOp));
    With explicit choice of a message operator like above, it works. Would be nice if you can explain why this is the case. Thanks.
    Thursday, January 22, 2015 10:48 AM
  • I just tried your code with version 2.6 and it works fine.
    • Marked as answer by wittawatj Thursday, January 29, 2015 1:02 PM
    Thursday, January 22, 2015 1:44 PM
    Owner
  • Thanks. This is strange. I am also using version 2.6. It must be because I have my custom message operators for Logistic in the path. Perhaps the inference engine picked my implementations.
    Thursday, January 22, 2015 1:57 PM