ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-24544][SQL][FOLLOWUP] Remove a wrong warning on Hive fallback lookup

Browse source 
## What changes were proposed in this pull request?

This PR is a follow-up of https://github.com/apache/spark/pull/21790 which causes a regression to show misleading warnings always at first invocation for all Hive function. Hive fallback lookup should not be warned. It's a normal process in function lookups.

**CURRENT (Showing `NoSuchFunctionException` and working)**
```scala
scala> sql("select histogram_numeric(a,2) from values(1) T(a)").show
19/06/02 22:02:10 WARN HiveSessionCatalog: Encountered a failure during looking up
function: org.apache.spark.sql.catalyst.analysis.NoSuchFunctionException:
Undefined function: 'histogram_numeric'. This function is neither a registered temporary
function nor a permanent function registered in the database 'default'.;
  at org.apache.spark.sql.catalyst.catalog.SessionCatalog.failFunctionLookup(SessionCatalog.scala:1234)
  at org.apache.spark.sql.catalyst.catalog.SessionCatalog.lookupFunction(SessionCatalog.scala:1302)
...
+------------------------+
|histogram_numeric( a, 2)|
+------------------------+
|            [[1.0, 1.0]]|
+------------------------+
```

## How was this patch tested?

Manually execute the above query.

Closes #24773 from dongjoon-hyun/SPARK-24544.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
This commit is contained in:
Dongjoon Hyun 2019-06-03 00:04:00 -07:00
parent 8b18ef5c7b
commit 8486680b34
1 changed files with 0 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala
View file

@ -142,8 +142,6 @@ private[sql] class HiveSessionCatalog(
// let's try to load it as a Hive's built-in function. // let's try to load it as a Hive's built-in function.
// Hive is case insensitive. // Hive is case insensitive.
val functionName = funcName.unquotedString.toLowerCase(Locale.ROOT) val functionName = funcName.unquotedString.toLowerCase(Locale.ROOT)
logWarning("Encountered a failure during looking up function:" +
s" ${Utils.exceptionString(error)}")
if (!hiveFunctions.contains(functionName)) { if (!hiveFunctions.contains(functionName)) {
failFunctionLookup(funcName, Some(error)) failFunctionLookup(funcName, Some(error))
} }