dd80845735
### What changes were proposed in this pull request? There is only one failed case of `streaming` module in Scala 2.13: `start with non-serializable DStream checkpoint ` in `StreamingContextSuite`. `StackOverflowError` is thrown here when `SerializationDebugger#visit` method is called. I found that `inputStreams` and `outputStreams` in `DStreamGraph` can not be matched in `SerializationDebugger#visit` method because `ArrayBuffer` in not `Array` in Scala 2.13. The main change of this pr is use `mutable.ArraySeq` instead of `ArrayBuffer` to store `inputStreams` and `outputStreams` in `DStreamGraph`, then it can be matched in `SerializationDebugger#visit` method. ### Why are the changes needed? We need to support a Scala 2.13 build. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? - Scala 2.12: Pass the Jenkins or GitHub Action - Scala 2.13: Pass GitHub 2.13 Build Action Do the following: ``` dev/change-scala-version.sh 2.13 mvn clean install -DskipTests -pl streaming -Pscala-2.13 -am mvn test -pl streaming -Pscala-2.13 mvn test -pl core -Pscala-2.13 ``` streaming module: ``` Tests: succeeded 339, failed 0, canceled 0, ignored 2, pending 0 All tests passed. ``` core module: ``` Tests: succeeded 2648, failed 0, canceled 4, ignored 7, pending 0 All tests passed. ``` Closes #29836 from LuciferYang/fix-streaming-213. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
---|---|---|
.. | ||
src | ||
pom.xml |