[SPARK-36279][INFRA][PYTHON] Fix lint-python to work with Python 3.9
### What changes were proposed in this pull request?
Fix `lint-python` to pick `PYTHON_EXECUTABLE` from the environment variable first to switch the Python and explicitly specify `PYTHON_EXECUTABLE` to use `python3.9` in CI.
### Why are the changes needed?
Currently `lint-python` uses `python3`, but it's not the one we expect in CI.
As a result, `black` check is not working.
```
The python3 -m black command was not found. Skipping black checks for now.
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
The `black` check in `lint-python` should work.
Closes #33507 from ueshin/issues/SPARK-36279/lint-python.
Authored-by: Takuya UESHIN <ueshin@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
(cherry picked from commit 663cbdfbe5
)
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
This commit is contained in:
parent
44cfce8548
commit
c1434b1928
2
.github/workflows/build_and_test.yml
vendored
2
.github/workflows/build_and_test.yml
vendored
|
@ -386,7 +386,7 @@ jobs:
|
|||
- name: Java linter
|
||||
run: ./dev/lint-java
|
||||
- name: Python linter
|
||||
run: ./dev/lint-python
|
||||
run: PYTHON_EXECUTABLE=python3.9 ./dev/lint-python
|
||||
- name: R linter
|
||||
run: ./dev/lint-r
|
||||
- name: JS linter
|
||||
|
|
|
@ -24,7 +24,7 @@ MYPY_BUILD="mypy"
|
|||
PYCODESTYLE_BUILD="pycodestyle"
|
||||
MINIMUM_PYCODESTYLE="2.7.0"
|
||||
|
||||
PYTHON_EXECUTABLE="python3"
|
||||
PYTHON_EXECUTABLE="${PYTHON_EXECUTABLE:-python3}"
|
||||
|
||||
BLACK_BUILD="$PYTHON_EXECUTABLE -m black"
|
||||
|
||||
|
|
|
@ -358,9 +358,7 @@ class NumOpsTest(PandasOnSparkTestCase, TestCasesUtils):
|
|||
for col in self.numeric_df_cols:
|
||||
pser, psser = pdf[col], psdf[col]
|
||||
if isinstance(psser.spark.data_type, DecimalType):
|
||||
self.assertRaisesRegex(
|
||||
TypeError, "< can not be applied to", lambda: psser < psser
|
||||
)
|
||||
self.assertRaisesRegex(TypeError, "< can not be applied to", lambda: psser < psser)
|
||||
else:
|
||||
self.assert_eq(pser < pser, psser < psser)
|
||||
|
||||
|
@ -380,9 +378,7 @@ class NumOpsTest(PandasOnSparkTestCase, TestCasesUtils):
|
|||
for col in self.numeric_df_cols:
|
||||
pser, psser = pdf[col], psdf[col]
|
||||
if isinstance(psser.spark.data_type, DecimalType):
|
||||
self.assertRaisesRegex(
|
||||
TypeError, "> can not be applied to", lambda: psser > psser
|
||||
)
|
||||
self.assertRaisesRegex(TypeError, "> can not be applied to", lambda: psser > psser)
|
||||
else:
|
||||
self.assert_eq(pser > pser, psser > psser)
|
||||
|
||||
|
|
Loading…
Reference in a new issue