ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-29137][ML][PYTHON][TESTS] Increase the timeout for StreamingLinearRegressionWithTests.test_train_prediction

Browse source 
### What changes were proposed in this pull request?

It increases the timeout for `StreamingLinearRegressionWithTests.test_train_prediction`

```
Traceback (most recent call last):
  File "/home/jenkins/workspace/SparkPullRequestBuilder3/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 503, in test_train_prediction
    self._eventually(condition)
  File "/home/jenkins/workspace/SparkPullRequestBuilder3/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 69, in _eventually
    lastValue = condition()
  File "/home/jenkins/workspace/SparkPullRequestBuilder3/python/pyspark/mllib/tests/test_streaming_algorithms.py", line 498, in condition
    self.assertGreater(errors[1] - errors[-1], 2)
AssertionError: 1.672640157855923 not greater than 2
```

This could likely happen when the PySpark tests run in parallel and it become slow.

### Why are the changes needed?

To make the tests less flaky. Seems it's being reported multiple times:

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123144/consoleFull
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123146/testReport/
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123141/testReport/
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123142/testReport/

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

Jenkins will test it out.

Closes #28701 from HyukjinKwon/SPARK-29137.

Authored-by: HyukjinKwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
HyukjinKwon 2020-06-01 22:03:47 -07:00 committed by Dongjoon Hyun
parent ff4a97d4b6
commit 64cb6f7066
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/pyspark/mllib/tests/test_streaming_algorithms.py
View file

@ -463,7 +463,7 @@ class StreamingLinearRegressionWithTests(MLLibStreamingTestCase):
return True
return "Latest errors: " + ", ".join(map(lambda x: str(x), errors))
eventually(condition)
eventually(condition, timeout=180.0)
if __name__ == "__main__":