8a3815f722
### What changes were proposed in this pull request?
Introducing a new test construct:
```
withHttpServer() { baseURL =>
...
}
```
Which starts and stops a Jetty server to serve files via HTTP.
Moreover this PR uses this new construct in the test `Run SparkRemoteFileTest using a remote data file`.
### Why are the changes needed?
Before this PR github URLs was used like "https://raw.githubusercontent.com/apache/spark/master/data/mllib/pagerank_data.txt".
This connects two Spark version in an unhealthy way like connecting the "master" branch which is moving part with the committed test code which is a non-moving (as it might be even released).
So this way a test running for an earlier version of Spark expects something (filename, content, path) from a the latter release and what is worse when the moving version is changed the earlier test will break.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing unit test.
Closes #31935 from attilapiros/SPARK-34789.
Authored-by: “attilapiros” <piros.attila.zsolt@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
|
||
|---|---|---|
| .. | ||
| kubernetes | ||
| mesos | ||
| yarn | ||