ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-6336] LBFGS should document what convergenceTol means

Browse source 
LBFGS uses convergence tolerance. This value should be written in document as an argument.

Author: lewuathe <lewuathe@me.com>

Closes #5033 from Lewuathe/SPARK-6336 and squashes the following commits:

e738b33 [lewuathe] Modify text to be more natural
ac03c3a [lewuathe] Modify documentations
6ccb304 [lewuathe] [SPARK-6336] LBFGS should document what convergenceTol means
This commit is contained in:
lewuathe 2015-03-17 12:11:57 -07:00 committed by Xiangrui Meng
parent 4cca3917dc
commit d9f3e01688
2 changed files with 9 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/mllib-optimization.md
View file

@ -203,6 +203,10 @@ regularization, as well as L2 regularizer.
recommended. recommended.
* `maxNumIterations` is the maximal number of iterations that L-BFGS can be run. * `maxNumIterations` is the maximal number of iterations that L-BFGS can be run.
* `regParam` is the regularization parameter when using regularization. * `regParam` is the regularization parameter when using regularization.
* `convergenceTol` controls how much relative change is still allowed when L-BFGS
is considered to converge. This must be nonnegative. Lower values are less tolerant and
therefore generally cause more iterations to be run. This value looks at both average
improvement and the norm of gradient inside [Breeze LBFGS](https://github.com/scalanlp/breeze/blob/master/math/src/main/scala/breeze/optimize/LBFGS.scala).
The `return` is a tuple containing two elements. The first element is a column matrix The `return` is a tuple containing two elements. The first element is a column matrix
containing weights for every feature, and the second element is an array containing containing weights for every feature, and the second element is an array containing

6
mllib/src/main/scala/org/apache/spark/mllib/optimization/LBFGS.scala
View file

@ -60,6 +60,8 @@ class LBFGS(private var gradient: Gradient, private var updater: Updater)
/** /**
* Set the convergence tolerance of iterations for L-BFGS. Default 1E-4. * Set the convergence tolerance of iterations for L-BFGS. Default 1E-4.
* Smaller value will lead to higher accuracy with the cost of more iterations. * Smaller value will lead to higher accuracy with the cost of more iterations.
* This value must be nonnegative. Lower convergence values are less tolerant
* and therefore generally cause more iterations to be run.
*/ */
def setConvergenceTol(tolerance: Double): this.type = { def setConvergenceTol(tolerance: Double): this.type = {
this.convergenceTol = tolerance this.convergenceTol = tolerance
@ -142,7 +144,9 @@ object LBFGS extends Logging {
* one single data example) * one single data example)
* @param updater - Updater function to actually perform a gradient step in a given direction. * @param updater - Updater function to actually perform a gradient step in a given direction.
* @param numCorrections - The number of corrections used in the L-BFGS update. * @param numCorrections - The number of corrections used in the L-BFGS update.
* @param convergenceTol - The convergence tolerance of iterations for L-BFGS * @param convergenceTol - The convergence tolerance of iterations for L-BFGS which is must be
* nonnegative. Lower values are less tolerant and therefore generally
* cause more iterations to be run.
* @param maxNumIterations - Maximal number of iterations that L-BFGS can be run. * @param maxNumIterations - Maximal number of iterations that L-BFGS can be run.
* @param regParam - Regularization parameter * @param regParam - Regularization parameter
* *