spark-instrumented-optimizer/python/pyspark/sql
hyukjinkwon 01dd008301 [SPARK-17764][SQL] Add to_json supporting to convert nested struct column to JSON string
## What changes were proposed in this pull request?

This PR proposes to add `to_json` function in contrast with `from_json` in Scala, Java and Python.

It'd be useful if we can convert a same column from/to json. Also, some datasources do not support nested types. If we are forced to save a dataframe into those data sources, we might be able to work around by this function.

The usage is as below:

``` scala
val df = Seq(Tuple1(Tuple1(1))).toDF("a")
df.select(to_json($"a").as("json")).show()
```

``` bash
+--------+
|    json|
+--------+
|{"_1":1}|
+--------+
```
## How was this patch tested?

Unit tests in `JsonFunctionsSuite` and `JsonExpressionsSuite`.

Author: hyukjinkwon <gurwls223@gmail.com>

Closes #15354 from HyukjinKwon/SPARK-17764.
2016-11-01 12:46:41 -07:00
..
__init__.py [SPARK-16772][PYTHON][DOCS] Fix API doc references to UDFRegistration + Update "important classes" 2016-08-06 05:02:59 +01:00
catalog.py [SPARK-17338][SQL][FOLLOW-UP] add global temp view 2016-10-11 15:21:28 +08:00
column.py [SPARK-17215][SQL] Method SQLContext.parseDataType(dataTypeString: String) could be removed. 2016-08-24 23:36:04 -07:00
conf.py [SPARK-15464][ML][MLLIB][SQL][TESTS] Replace SQLContext and SparkContext with SparkSession using builder pattern in python test code 2016-05-23 18:14:48 -07:00
context.py [SPARK-11775][PYSPARK][SQL] Allow PySpark to register Java UDF 2016-10-14 15:50:35 -07:00
dataframe.py [SPARK-17946][PYSPARK] Python crossJoin API similar to Scala 2016-10-14 18:24:47 -07:00
functions.py [SPARK-17764][SQL] Add to_json supporting to convert nested struct column to JSON string 2016-11-01 12:46:41 -07:00
group.py [MINOR][PYSPARK][DOC] Fix wrongly formatted examples in PySpark documentation 2016-07-06 10:45:51 -07:00
readwriter.py [SPARK-17764][SQL] Add to_json supporting to convert nested struct column to JSON string 2016-11-01 12:46:41 -07:00
session.py [SPARK-17720][SQL] introduce static SQL conf 2016-10-11 20:27:08 -07:00
streaming.py [SPARK-17764][SQL] Add to_json supporting to convert nested struct column to JSON string 2016-11-01 12:46:41 -07:00
tests.py [SPARK-17946][PYSPARK] Python crossJoin API similar to Scala 2016-10-14 18:24:47 -07:00
types.py [SPARK-17215][SQL] Method SQLContext.parseDataType(dataTypeString: String) could be removed. 2016-08-24 23:36:04 -07:00
utils.py [SPARK-15953][WIP][STREAMING] Renamed ContinuousQuery to StreamingQuery 2016-06-15 10:46:07 -07:00
window.py [SPARK-17845] [SQL] More self-evident window function frame boundary API 2016-10-12 16:45:10 -07:00