[SPARK-27486][CORE][TEST] Enable History server storage information test in the HistoryServerSuite
## What changes were proposed in this pull request? We have disabled a test related to storage in the History server suite after SPARK-13845. But, after SPARK-22050, we can store the information about block updated events to eventLog, if we enable "spark.eventLog.logBlockUpdates.enabled=true". So, we can enable the test, by adding an eventlog corresponding to the application, which has enabled the configuration, "spark.eventLog.logBlockUpdates.enabled=true" ## How was this patch tested? Existing UTs Closes #24390 from shahidki31/enableRddStorageTest. Authored-by: Shahid <shahidki31@gmail.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
This commit is contained in:
parent
31488e1ca5
commit
16bbe0f798
|
@ -0,0 +1,10 @@
|
|||
{
|
||||
"id" : 0,
|
||||
"name" : "0",
|
||||
"numPartitions" : 8,
|
||||
"numCachedPartitions" : 0,
|
||||
"storageLevel" : "Memory Deserialized 1x Replicated",
|
||||
"memoryUsed" : 0,
|
||||
"diskUsed" : 0,
|
||||
"partitions" : [ ]
|
||||
}
|
|
@ -174,9 +174,11 @@ class HistoryServerSuite extends SparkFunSuite with BeforeAndAfter with Matchers
|
|||
"executor node blacklisting unblacklisting" -> "applications/app-20161115172038-0000/executors",
|
||||
"executor memory usage" -> "applications/app-20161116163331-0000/executors",
|
||||
|
||||
"app environment" -> "applications/app-20161116163331-0000/environment"
|
||||
// Todo: enable this test when logging the even of onBlockUpdated. See: SPARK-13845
|
||||
// "one rdd storage json" -> "applications/local-1422981780767/storage/rdd/0"
|
||||
"app environment" -> "applications/app-20161116163331-0000/environment",
|
||||
|
||||
// Enable "spark.eventLog.logBlockUpdates.enabled", to get the storage information
|
||||
// in the history server.
|
||||
"one rdd storage json" -> "applications/local-1422981780767/storage/rdd/0"
|
||||
)
|
||||
|
||||
// run a bunch of characterization tests -- just verify the behavior is the same as what is saved
|
||||
|
|
Loading…
Reference in a new issue