ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-9869] [STREAMING] Wait for all event notifications before asserting results

Browse source 
Author: robbins <robbins@uk.ibm.com>

Closes #8589 from robbinspg/InputStreamSuite-fix.
This commit is contained in:
robbins 2015-09-03 13:48:35 -07:00 committed by Andrew Or
parent d911c682f0
commit 754f853b02
1 changed files with 3 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala
View file

@ -76,6 +76,9 @@ class InputStreamsSuite extends TestSuiteBase with BeforeAndAfter {
fail("Timeout: cannot finish all batches in 30 seconds") fail("Timeout: cannot finish all batches in 30 seconds")
} }
// Ensure progress listener has been notified of all events
ssc.scheduler.listenerBus.waitUntilEmpty(500)
// Verify all "InputInfo"s have been reported // Verify all "InputInfo"s have been reported
assert(ssc.progressListener.numTotalReceivedRecords === input.size) assert(ssc.progressListener.numTotalReceivedRecords === input.size)
assert(ssc.progressListener.numTotalProcessedRecords === input.size) assert(ssc.progressListener.numTotalProcessedRecords === input.size)