[SPARK-16470][ML][OPTIMIZER] Check linear regression training whether actually reach convergence and add warning if not
## What changes were proposed in this pull request? In `ml.regression.LinearRegression`, it use breeze `LBFGS` and `OWLQN` optimizer to do data training, but do not check whether breeze's optimizer returned result actually reached convergence. The `LBFGS` and `OWLQN` optimizer in breeze finish iteration may result the following situations: 1) reach max iteration number 2) function reach value convergence 3) objective function stop improving 4) gradient reach convergence 5) search failed(due to some internal numerical error) I add warning printing code so that if the iteration result is (1) or (3) or (5) in above, it will print a warning with respective reason string. ## How was this patch tested? Manual. Author: WeichenXu <WeichenXu123@outlook.com> Closes #14122 from WeichenXu123/add_lr_not_convergence_warn.
This commit is contained in:
parent
5b28e02584
commit
6cb75db9ab
|
@ -327,6 +327,11 @@ class LinearRegression @Since("1.3.0") (@Since("1.3.0") override val uid: String
|
|||
throw new SparkException(msg)
|
||||
}
|
||||
|
||||
if (!state.actuallyConverged) {
|
||||
logWarning("LinearRegression training fininshed but the result " +
|
||||
s"is not converged because: ${state.convergedReason.get.reason}")
|
||||
}
|
||||
|
||||
/*
|
||||
The coefficients are trained in the scaled space; we're converting them back to
|
||||
the original space.
|
||||
|
|
Loading…
Reference in a new issue