[SPARK-7684] [SQL] Refactoring MetastoreDataSourcesSuite to workaround SPARK-7684

As stated in SPARK-7684, currently `TestHive.reset` has some execution order specific bug, which makes running specific test suites locally pretty frustrating. This PR refactors `MetastoreDataSourcesSuite` (which relies on `TestHive.reset` heavily) using various `withXxx` utility methods in `SQLTestUtils` to ask each test case to cleanup their own mess so that we can avoid calling `TestHive.reset`.

Author: Cheng Lian <lian@databricks.com>
Author: Yin Huai <yhuai@databricks.com>

Closes #6353 from liancheng/workaround-spark-7684 and squashes the following commits:

26939aa [Yin Huai] Move the initialization of jsonFilePath to beforeAll.
a423d48 [Cheng Lian] Fixes Scala style issue
dfe45d0 [Cheng Lian] Refactors MetastoreDataSourcesSuite to workaround SPARK-7684
92a116d [Cheng Lian] Fixes minor styling issues

(cherry picked from commit b97ddff000)
Signed-off-by: Yin Huai <yhuai@databricks.com>
This commit is contained in:
Cheng Lian 2015-05-27 13:09:33 -07:00 committed by Yin Huai
parent d33142fd8c
commit 89fe93fc3b
3 changed files with 658 additions and 602 deletions

View file

@ -67,6 +67,10 @@ class QueryTest extends PlanTest {
checkAnswer(df, Seq(expectedAnswer)) checkAnswer(df, Seq(expectedAnswer))
} }
protected def checkAnswer(df: DataFrame, expectedAnswer: DataFrame): Unit = {
checkAnswer(df, expectedAnswer.collect())
}
def sqlTest(sqlString: String, expectedAnswer: Seq[Row])(implicit sqlContext: SQLContext) { def sqlTest(sqlString: String, expectedAnswer: Seq[Row])(implicit sqlContext: SQLContext) {
test(sqlString) { test(sqlString) {
checkAnswer(sqlContext.sql(sqlString), expectedAnswer) checkAnswer(sqlContext.sql(sqlString), expectedAnswer)

View file

@ -75,14 +75,18 @@ trait SQLTestUtils {
/** /**
* Drops temporary table `tableName` after calling `f`. * Drops temporary table `tableName` after calling `f`.
*/ */
protected def withTempTable(tableName: String)(f: => Unit): Unit = { protected def withTempTable(tableNames: String*)(f: => Unit): Unit = {
try f finally sqlContext.dropTempTable(tableName) try f finally tableNames.foreach(sqlContext.dropTempTable)
} }
/** /**
* Drops table `tableName` after calling `f`. * Drops table `tableName` after calling `f`.
*/ */
protected def withTable(tableName: String)(f: => Unit): Unit = { protected def withTable(tableNames: String*)(f: => Unit): Unit = {
try f finally sqlContext.sql(s"DROP TABLE IF EXISTS $tableName") try f finally {
tableNames.foreach { name =>
sqlContext.sql(s"DROP TABLE IF EXISTS $name")
}
}
} }
} }