locked
convergence condition of LBFGS/Quasi Newton RRS feed

  • Question

  • I found the LBFGS optimizer (MicrosoftResearch.Infer.Maths.LBFGS) is a great tool. In practice, however, it seems the convergence condition/eps of this class is rather rigorous. While I do not need so rigorous convergence condition and I want the optimizer to stop quickly, I am not sureif I can set the stopping/convergence condition, or eps by myself?

    Thanks!

    Friday, August 8, 2014 6:52 AM

Answers

  • There are several options/considerations for convergence. You can both set an upper limit for the number of iterations, and set an epsilon, and it will stop with whichever condition gets met first. As regards the epsilon, you can choose whether to use either the objective (|F(k+1)-F(k)| <= epsilon*max{|F(k)|,|F(k+1)|,1}) or the gradient (|grad F|/sqrt(dimension) <= epsilon) to determine convergence.

    Here is what the code looks like for these options: 

    LBFGS optimizer = new LBFGS(5);
    optimizer.Epsilon = 0.001;
    optimizer.convergenceCriteria = BFGS.ConvergenceCriteria.Gradient;
    // optimizer.convergenceCriteria = BFGS.ConvergenceCriteria.Objective;
    optimizer.MaximumIterations = 50;

    John

    Friday, August 8, 2014 7:16 AM
    Owner

All replies

  • There are several options/considerations for convergence. You can both set an upper limit for the number of iterations, and set an epsilon, and it will stop with whichever condition gets met first. As regards the epsilon, you can choose whether to use either the objective (|F(k+1)-F(k)| <= epsilon*max{|F(k)|,|F(k+1)|,1}) or the gradient (|grad F|/sqrt(dimension) <= epsilon) to determine convergence.

    Here is what the code looks like for these options: 

    LBFGS optimizer = new LBFGS(5);
    optimizer.Epsilon = 0.001;
    optimizer.convergenceCriteria = BFGS.ConvergenceCriteria.Gradient;
    // optimizer.convergenceCriteria = BFGS.ConvergenceCriteria.Objective;
    optimizer.MaximumIterations = 50;

    John

    Friday, August 8, 2014 7:16 AM
    Owner
  • Thanks a lot, John!

    Friday, August 8, 2014 8:22 PM