ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/python/pyspark/sql
History
Yin Huai 994abbaeb3 [SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode 
https://issues.apache.org/jira/browse/SPARK-8532

This PR has two changes. First, it fixes the bug that save actions (i.e. `save/saveAsTable/json/parquet/jdbc`) always override mode. Second, it adds input argument `partitionBy` to `save/saveAsTable/parquet`.

Author: Yin Huai <yhuai@databricks.com>

Closes #6937 from yhuai/SPARK-8532 and squashes the following commits:

f972d5d [Yin Huai] davies's comment.
d37abd2 [Yin Huai] style.
d21290a [Yin Huai] Python doc.
889eb25 [Yin Huai] Minor refactoring and add partitionBy to save, saveAsTable, and parquet.
7fbc24b [Yin Huai] Use None instead of "error" as the default value of mode since JVM-side already uses "error" as the default value.
d696dff [Yin Huai] Python style.
88eb6c4 [Yin Huai] If mode is "error", do not call mode method.
c40c461 [Yin Huai] Regression test.

(cherry picked from commit 5ab9fcfb01)
Signed-off-by: Yin Huai <yhuai@databricks.com>
2015-06-22 13:51:34 -07:00
..
__init__.py [SPARK-8060] Improve DataFrame Python test coverage and documentation. 2015-06-03 00:23:42 -07:00
column.py [SPARK-8038] [SQL] [PYSPARK] fix Column.when() and otherwise() 2015-06-02 13:38:14 -07:00
context.py [SPARK-7980] [SQL] Support SQLContext.range(end) 2015-06-03 11:28:38 -07:00
dataframe.py [SPARK-8146] DataFrame Python API: Alias replace in df.na 2015-06-07 01:21:08 -07:00
functions.py [SPARK-7322, SPARK-7836, SPARK-7822][SQL] DataFrame window function related updates 2015-05-23 08:30:18 -07:00
group.py [SPARK-7322, SPARK-7836, SPARK-7822][SQL] DataFrame window function related updates 2015-05-23 08:30:18 -07:00
readwriter.py [SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode 2015-06-22 13:51:34 -07:00
tests.py [SPARK-8532] [SQL] In Python's DataFrameWriter, save/saveAsTable/json/parquet/jdbc always override mode 2015-06-22 13:51:34 -07:00
types.py [SPARK-7899] [PYSPARK] Fix Python 3 pyspark/sql/types module conflict 2015-06-01 16:56:04 -07:00
window.py [SPARK-8146] DataFrame Python API: Alias replace in df.na 2015-06-07 01:21:08 -07:00