ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/python/pyspark/sql
History
Yuanjian Li dbb4d83829 [SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark 
## What changes were proposed in this pull request?

Implement `_repr_html_` for PySpark while in notebook and add config named "spark.sql.repl.eagerEval.enabled" to control this.

The dev list thread for context: http://apache-spark-developers-list.1001551.n3.nabble.com/eager-execution-and-debuggability-td23928.html

## How was this patch tested?

New ut in DataFrameSuite and manual test in jupyter. Some screenshot below.

**After:**
![image](https://user-images.githubusercontent.com/4833765/40268422-8db5bef0-5b9f-11e8-80f1-04bc654a4f2c.png)

**Before:**
![image](https://user-images.githubusercontent.com/4833765/40268431-9f92c1b8-5b9f-11e8-9db9-0611f0940b26.png)

Author: Yuanjian Li <xyliyuanjian@gmail.com>

Closes #21370 from xuanyuanking/SPARK-24215.
2018-06-05 08:23:08 +07:00
..
__init__.py [SPARK-22369][PYTHON][DOCS] Exposes catalog API documentation in PySpark 2017-11-02 15:22:52 +01:00
catalog.py [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
column.py [SPARK-23847][PYTHON][SQL] Add asc_nulls_first, asc_nulls_last to PySpark 2018-04-08 12:09:06 +08:00
conf.py [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql 2018-03-26 12:42:32 +09:00
context.py [SPARK-23706][PYTHON] spark.conf.get(value, default=None) should produce None in PySpark 2018-03-18 20:24:14 +09:00
dataframe.py [SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark 2018-06-05 08:23:08 +07:00
functions.py [SPARK-23920][SQL] add array_remove to remove all elements that equal element from array 2018-05-31 22:04:26 -07:00
group.py [SPARK-24392][PYTHON] Label pandas_udf as Experimental 2018-05-28 12:56:05 +08:00
readwriter.py [SPARK-23786][SQL] Checking column names of csv headers 2018-06-03 22:02:21 -07:00
session.py [SPARK-24392][PYTHON] Label pandas_udf as Experimental 2018-05-28 12:56:05 +08:00
streaming.py [SPARK-23786][SQL] Checking column names of csv headers 2018-06-03 22:02:21 -07:00
tests.py [SPARK-24215][PYSPARK] Implement _repr_html_ for dataframes in PySpark 2018-06-05 08:23:08 +07:00
types.py [SPARK-24057][PYTHON] put the real data type in the AssertionError message 2018-04-26 14:21:22 -07:00
udf.py [SPARK-23754][PYTHON] Re-raising StopIteration in client code 2018-05-30 18:11:33 +08:00
utils.py [SPARK-23699][PYTHON][SQL] Raise same type of error caught with Arrow enabled 2018-03-27 20:06:12 -07:00
window.py [SPARK-23861][SQL][DOC] Clarify default window frame with and without orderBy clause 2018-04-07 00:15:54 +08:00