[SPARK-12314][SQL] isnull operator not pushed down for JDBC datasource.
https://issues.apache.org/jira/browse/SPARK-12314 `IsNull` filter is not being pushed down for JDBC datasource. It looks it is SQL standard according to [SQL-92](http://www.contrib.andrew.cmu.edu/~shadow/sql/sql1992.txt), SQL:1999, [SQL:2003](http://www.wiscorp.com/sql_2003_standard.zip) and [SQL:201x](http://www.wiscorp.com/sql20nn.zip) and I believe most databases support this. In this PR, I simply added the case for `IsNull` filter to produce a proper filter string. Author: hyukjinkwon <gurwls223@gmail.com> This patch had conflicts when merged, resolved by Committer: Reynold Xin <rxin@databricks.com> Closes #10286 from HyukjinKwon/SPARK-12314.
This commit is contained in:
parent
0f6936b5f1
commit
7f443a6879
|
@ -286,6 +286,7 @@ private[sql] class JDBCRDD(
|
|||
case GreaterThan(attr, value) => s"$attr > ${compileValue(value)}"
|
||||
case LessThanOrEqual(attr, value) => s"$attr <= ${compileValue(value)}"
|
||||
case GreaterThanOrEqual(attr, value) => s"$attr >= ${compileValue(value)}"
|
||||
case IsNull(attr) => s"$attr IS NULL"
|
||||
case _ => null
|
||||
}
|
||||
|
||||
|
|
|
@ -182,6 +182,7 @@ class JDBCSuite extends SparkFunSuite with BeforeAndAfter with SharedSQLContext
|
|||
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME = 'fred'")).collect().size === 1)
|
||||
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME > 'fred'")).collect().size === 2)
|
||||
assert(stripSparkFilter(sql("SELECT * FROM foobar WHERE NAME != 'fred'")).collect().size === 2)
|
||||
assert(stripSparkFilter(sql("SELECT * FROM nulltypes WHERE A IS NULL")).collect().size === 1)
|
||||
}
|
||||
|
||||
test("SELECT * WHERE (quoted strings)") {
|
||||
|
|
Loading…
Reference in a new issue