ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-32389][TESTS] Add all hive.execution suite in the parallel test group

Browse source 
### What changes were proposed in this pull request?

Add a new parallel test group for all `hive.execution` suites.

### Why are the changes needed?

Base on the tests, it can reduce the Jenkins testing time.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Existing tests.

Closes #28977 from xuanyuanking/parallelTest.

Authored-by: Yuanjian Li <xyliyuanjian@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
Yuanjian Li 2020-07-23 21:14:36 +09:00 committed by HyukjinKwon
parent 7b66882c9d
commit a71233f89d
1 changed files with 5 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
project/SparkBuild.scala
View file

@ -469,7 +469,6 @@ object SparkParallelTestGrouping {
"org.apache.spark.sql.catalyst.expressions.MathExpressionsSuite",
"org.apache.spark.sql.hive.HiveExternalCatalogSuite",
"org.apache.spark.sql.hive.StatisticsSuite",
"org.apache.spark.sql.hive.execution.HiveCompatibilitySuite",
"org.apache.spark.sql.hive.client.VersionsSuite",
"org.apache.spark.sql.hive.client.HiveClientVersions",
"org.apache.spark.sql.hive.HiveExternalCatalogVersionsSuite",
@ -486,9 +485,14 @@ object SparkParallelTestGrouping {
)
private val DEFAULT_TEST_GROUP = "default_test_group"
private val HIVE_EXECUTION_TEST_GROUP = "hive_execution_test_group"
private def testNameToTestGroup(name: String): String = name match {
case _ if testsWhichShouldRunInTheirOwnDedicatedJvm.contains(name) => name
// Different with the cases in testsWhichShouldRunInTheirOwnDedicatedJvm, here we are grouping
// all suites of `org.apache.spark.sql.hive.execution.*` into a single group, instead of
// launching one JVM per suite.
case _ if name.contains("org.apache.spark.sql.hive.execution") => HIVE_EXECUTION_TEST_GROUP
case _ => DEFAULT_TEST_GROUP
}