630dfdf550
## What changes were proposed in this pull request? It seems that https://bugs.openjdk.java.net/browse/JDK-8068730 makes `InputStreamsSuite` very flaky. <img width="903" alt="error" src="https://user-images.githubusercontent.com/9700541/59727067-017eb780-91e9-11e9-8bb0-ac5f4c1bc44d.png"> As we can see the Jenkins result, this can be reproduced frequently with JDK9+. ``` $ build/sbt "streaming/testOnly *.InputStreamsSuite" [info] - Modified files are correctly detected. *** FAILED *** (134 milliseconds) [info] Set("renamed") did not equal Set() (InputStreamsSuite.scala:312) [info] org.scalatest.exceptions.TestFailedException: ``` The reason is the `renamed.txt`'s modification time becomes greater than the clock in JDK9+ and Spark ignored it with **not selected** message. In JDK8, the modification time generated by this test case doesn't have `milliseconds` part. ``` Getting new files for time 1560896662000, ignoring files older than 1560896659679 file:/.../streaming/subdir/renamed.txt not selected as mod time 1560896662679 > current time 1560896662000 file:/.../streaming/subdir/existing ignored as mod time 1560896657679 <= ignore time 1560896659679 Finding new files took 0 ms New files at time 1560896662000 ms: ``` ## How was this patch tested? Pass the Jenkins and manually repeat the following with JDK11 10 times. ``` $ build/sbt "streaming/testOnly *.InputStreamsSuite" ``` Closes #24904 from dongjoon-hyun/SPARK-28101. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
|---|---|---|
| .. | ||
| src | ||
| pom.xml | ||
