ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[HOTFIX][SQL] Remove cleaning of UDFs

Browse source 
It is not safe to run the closure cleaner on slaves.  #2153 introduced this which broke all UDF execution on slaves.  Will re-add cleaning of UDF closures in a follow-up PR.

Author: Michael Armbrust <michael@databricks.com>

Closes #2174 from marmbrus/fixUdfs and squashes the following commits:

55406de [Michael Armbrust] [HOTFIX] Remove cleaning of UDFs
This commit is contained in:
Michael Armbrust 2014-08-27 23:05:34 -07:00 committed by Patrick Wendell
parent dafe343499
commit 024178c574
1 changed files with 0 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ScalaUdf.scala
View file

@ -23,9 +23,6 @@ import org.apache.spark.util.ClosureCleaner
case class ScalaUdf(function: AnyRef, dataType: DataType, children: Seq[Expression])
extends Expression {
// Clean function when not called with default no-arg constructor.
if (function != null) { ClosureCleaner.clean(function) }
type EvaluatedType = Any
def nullable = true