1c403733b8
## What changes were proposed in this pull request?
This PR corrects SparkR to use `shell()` instead of `system2()` on Windows.
Using `system2(...)` on Windows does not process windows file separator `\`. `shell(tralsate = TRUE, ...)` can treat this problem. So, this was changed to be chosen according to OS.
Existing tests were failed on Windows due to this problem. For example, those were failed.
```
8. Failure: sparkJars tag in SparkContext (test_includeJAR.R#34)
9. Failure: sparkJars tag in SparkContext (test_includeJAR.R#36)
```
The cases above were due to using of `system2`.
In addition, this PR also fixes some tests failed on Windows.
```
5. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#128)
6. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#131)
7. Failure: sparkJars sparkPackages as comma-separated strings (test_context.R#134)
```
The cases above were due to a weird behaviour of `normalizePath()`. On Linux, if the path does not exist, it just prints out the input but it prints out including the current path on Windows.
```r
# On Linus
path <- normalizePath("aa")
print(path)
[1] "aa"
# On Windows
path <- normalizePath("aa")
print(path)
[1] "C:\\Users\\aa"
```
## How was this patch tested?
Jenkins tests and manually tested in a Window machine as below:
Here is the [stdout](https://gist.github.com/HyukjinKwon/4bf35184f3a30f3bce987a58ec2bbbab) of testing.
Closes #7025
Author: hyukjinkwon <gurwls223@gmail.com>
Author: Hyukjin Kwon <gurwls223@gmail.com>
Author: Prakash PC <prakash.chinnu@gmail.com>
Closes #13165 from HyukjinKwon/pr/7025.
37 lines
1.5 KiB
R
37 lines
1.5 KiB
R
#
|
|
# Licensed to the Apache Software Foundation (ASF) under one or more
|
|
# contributor license agreements. See the NOTICE file distributed with
|
|
# this work for additional information regarding copyright ownership.
|
|
# The ASF licenses this file to You under the Apache License, Version 2.0
|
|
# (the "License"); you may not use this file except in compliance with
|
|
# the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
#
|
|
context("include an external JAR in SparkContext")
|
|
|
|
runScript <- function() {
|
|
sparkHome <- Sys.getenv("SPARK_HOME")
|
|
sparkTestJarPath <- "R/lib/SparkR/test_support/sparktestjar_2.10-1.0.jar"
|
|
jarPath <- paste("--jars", shQuote(file.path(sparkHome, sparkTestJarPath)))
|
|
scriptPath <- file.path(sparkHome, "R/lib/SparkR/tests/testthat/jarTest.R")
|
|
submitPath <- file.path(sparkHome, paste("bin/", determineSparkSubmitBin(), sep = ""))
|
|
combinedArgs <- paste(jarPath, scriptPath, sep = " ")
|
|
res <- launchScript(submitPath, combinedArgs, capture = TRUE)
|
|
tail(res, 2)
|
|
}
|
|
|
|
test_that("sparkJars tag in SparkContext", {
|
|
testOutput <- runScript()
|
|
helloTest <- testOutput[1]
|
|
expect_equal(helloTest, "Hello, Dave")
|
|
basicFunction <- testOutput[2]
|
|
expect_equal(basicFunction, "4")
|
|
})
|