ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[PYSPARK][DOCS][MINOR] Changed :func: to :attr: Sphinx roles, fixed links in documentation of Data{Frame,Stream}{Reader,Writer}

Browse source 
This commit is published into the public domain.

### What changes were proposed in this pull request?
This PR fixes the documentation of `DataFrameReader`, `DataFrameWriter`, `DataStreamReader`, and `DataStreamWriter`, where attributes of other classes were misrepresented as functions. Additionally, creation of hyperlinks across modules was fixed in these instances.

### Why are the changes needed?
The old state produced documentation that suggested invalid usage of PySpark objects (accessing attributes as though they were callable.)

### Does this PR introduce any user-facing change?
No, except for improved documentation.

### How was this patch tested?
No test added; documentation build runs through.

Closes #27553 from DavidToneian/docfix-DataFrameReader-DataFrameWriter-DataStreamReader-DataStreamWriter.

Authored-by: David Toneian <david@toneian.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
David Toneian 2020-02-14 11:00:35 +09:00 committed by HyukjinKwon
parent fa3517cdb1
commit 25db8c71a2
2 changed files with 8 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
python/pyspark/sql/readwriter.py
View file

@ -48,7 +48,7 @@ class OptionUtils(object):
class DataFrameReader(OptionUtils): class DataFrameReader(OptionUtils):
""" """
Interface used to load a :class:`DataFrame` from external storage systems Interface used to load a :class:`DataFrame` from external storage systems
(e.g. file systems, key-value stores, etc). Use :func:`spark.read` (e.g. file systems, key-value stores, etc). Use :attr:`SparkSession.read`
to access this. to access this.
.. versionadded:: 1.4 .. versionadded:: 1.4
@ -616,7 +616,7 @@ class DataFrameReader(OptionUtils):
class DataFrameWriter(OptionUtils): class DataFrameWriter(OptionUtils):
""" """
Interface used to write a :class:`DataFrame` to external storage systems Interface used to write a :class:`DataFrame` to external storage systems
(e.g. file systems, key-value stores, etc). Use :func:`DataFrame.write` (e.g. file systems, key-value stores, etc). Use :attr:`DataFrame.write`
to access this. to access this.
.. versionadded:: 1.4 .. versionadded:: 1.4

11
python/pyspark/sql/streaming.py
View file

@ -276,9 +276,9 @@ class StreamingQueryManager(object):
class DataStreamReader(OptionUtils): class DataStreamReader(OptionUtils):
""" """
Interface used to load a streaming :class:`DataFrame` from external storage systems Interface used to load a streaming :class:`DataFrame <pyspark.sql.DataFrame>` from external
(e.g. file systems, key-value stores, etc). Use :func:`spark.readStream` storage systems (e.g. file systems, key-value stores, etc).
to access this. Use :attr:`SparkSession.readStream <pyspark.sql.SparkSession.readStream>` to access this.
.. note:: Evolving. .. note:: Evolving.
@ -750,8 +750,9 @@ class DataStreamReader(OptionUtils):
class DataStreamWriter(object): class DataStreamWriter(object):
""" """
Interface used to write a streaming :class:`DataFrame` to external storage systems Interface used to write a streaming :class:`DataFrame <pyspark.sql.DataFrame>` to external
(e.g. file systems, key-value stores, etc). Use :func:`DataFrame.writeStream` storage systems (e.g. file systems, key-value stores, etc).
Use :attr:`DataFrame.writeStream <pyspark.sql.DataFrame.writeStream>`
to access this. to access this.
.. note:: Evolving. .. note:: Evolving.