spark-instrumented-optimizer/python/pyspark/sql
Takeshi Yamamuro b59808385c [SPARK-23023][SQL] Cast field data to strings in showString
## What changes were proposed in this pull request?
The current `Datset.showString` prints rows thru `RowEncoder` deserializers like;
```
scala> Seq(Seq(Seq(1, 2), Seq(3), Seq(4, 5, 6))).toDF("a").show(false)
+------------------------------------------------------------+
|a                                                           |
+------------------------------------------------------------+
|[WrappedArray(1, 2), WrappedArray(3), WrappedArray(4, 5, 6)]|
+------------------------------------------------------------+
```
This result is incorrect because the correct one is;
```
scala> Seq(Seq(Seq(1, 2), Seq(3), Seq(4, 5, 6))).toDF("a").show(false)
+------------------------+
|a                       |
+------------------------+
|[[1, 2], [3], [4, 5, 6]]|
+------------------------+
```
So, this pr fixed code in `showString` to cast field data to strings before printing.

## How was this patch tested?
Added tests in `DataFrameSuite`.

Author: Takeshi Yamamuro <yamamuro@apache.org>

Closes #20214 from maropu/SPARK-23023.
2018-01-15 16:26:52 +08:00
..
__init__.py [SPARK-22369][PYTHON][DOCS] Exposes catalog API documentation in PySpark 2017-11-02 15:22:52 +01:00
catalog.py [SPARK-22939][PYSPARK] Support Spark UDF in registerFunction 2018-01-04 21:07:31 +08:00
column.py [SPARK-19165][PYTHON][SQL] PySpark APIs using columns as arguments should validate input types for column 2017-08-24 20:29:03 +09:00
conf.py [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code 2016-05-23 18:14:48 -07:00
context.py [SPARK-22939][PYSPARK] Support Spark UDF in registerFunction 2018-01-04 21:07:31 +08:00
dataframe.py [SPARK-22874][PYSPARK][SQL] Modify checking pandas version to use LooseVersion. 2017-12-22 20:09:51 +09:00
functions.py [SPARK-23023][SQL] Cast field data to strings in showString 2018-01-15 16:26:52 +08:00
group.py [SPARK-22324][SQL][PYTHON] Upgrade Arrow to 0.8.0 2017-12-21 20:43:56 +09:00
readwriter.py [SPARK-22818][SQL] csv escape of quote escape 2017-12-29 07:30:06 +08:00
session.py [SPARK-23009][PYTHON] Fix for non-str col names to createDataFrame from Pandas 2018-01-10 14:55:24 +09:00
streaming.py [SPARK-22933][SPARKR] R Structured Streaming API for withWatermark, trigger, partitionBy 2018-01-03 21:43:14 -08:00
tests.py [SPARK-23009][PYTHON] Fix for non-str col names to createDataFrame from Pandas 2018-01-10 14:55:24 +09:00
types.py [SPARK-22566][PYTHON] Better error message for _merge_type in Pandas to Spark DF conversion 2018-01-08 14:32:05 +09:00
udf.py [SPARK-22901][PYTHON][FOLLOWUP] Adds the doc for asNondeterministic for wrapped UDF function 2018-01-06 23:08:26 +08:00
utils.py [SPARK-22874][PYSPARK][SQL][FOLLOW-UP] Modify error messages to show actual versions. 2017-12-25 20:29:10 +09:00
window.py [SPARK-18690][PYTHON][SQL] Backward compatibility of unbounded frames 2016-12-02 17:39:28 -08:00