spark-instrumented-optimizer/python/pyspark/sql
Reynold Xin 9952217749 [SPARK-10731] [SQL] Delegate to Scala's DataFrame.take implementation in Python DataFrame.
Python DataFrame.head/take now requires scanning all the partitions. This pull request changes them to delegate the actual implementation to Scala DataFrame (by calling DataFrame.take).

This is more of a hack for fixing this issue in 1.5.1. A more proper fix is to change executeCollect and executeTake to return InternalRow rather than Row, and thus eliminate the extra round-trip conversion.

Author: Reynold Xin <rxin@databricks.com>

Closes #8876 from rxin/SPARK-10731.
2015-09-23 16:43:21 -07:00
..
__init__.py [SPARK-10373] [PYSPARK] move @since into pyspark from sql 2015-09-08 20:56:22 -07:00
column.py [SPARK-9014] [SQL] Allow Python spark API to use built-in exponential operator 2015-09-11 15:19:04 -07:00
context.py [SPARK-10373] [PYSPARK] move @since into pyspark from sql 2015-09-08 20:56:22 -07:00
dataframe.py [SPARK-10731] [SQL] Delegate to Scala's DataFrame.take implementation in Python DataFrame. 2015-09-23 16:43:21 -07:00
functions.py [SPARK-10577] [PYSPARK] DataFrame hint for broadcast join 2015-09-21 23:36:41 -07:00
group.py [SPARK-10373] [PYSPARK] move @since into pyspark from sql 2015-09-08 20:56:22 -07:00
readwriter.py [SPARK-10373] [PYSPARK] move @since into pyspark from sql 2015-09-08 20:56:22 -07:00
tests.py [SPARK-10577] [PYSPARK] DataFrame hint for broadcast join 2015-09-21 23:36:41 -07:00
types.py [SPARK-7544] [SQL] [PySpark] pyspark.sql.types.Row implements __getitem__ 2015-09-10 13:54:20 -07:00
utils.py [SPARK-9166][SQL][PYSPARK] Capture and hide IllegalArgumentException in Python API 2015-07-19 00:32:56 -07:00
window.py [SPARK-10373] [PYSPARK] move @since into pyspark from sql 2015-09-08 20:56:22 -07:00