[SPARK-31319][SQL][FOLLOW-UP] Add a SQL example for UDAF

### What changes were proposed in this pull request?
Add a SQL example for UDAF

### Why are the changes needed?
To make SQL Reference complete

### Does this PR introduce any user-facing change?
Yes.
Add the following page, also change ```Sql``` to ```SQL``` in the example tab for all the sql examples.
<img width="1110" alt="Screen Shot 2020-04-13 at 6 09 24 PM" src="https://user-images.githubusercontent.com/13592258/79175240-06cd7400-7db2-11ea-8f3e-af71a591a64b.png">

### How was this patch tested?
Manually build and check

Closes #28209 from huaxingao/udf_followup.

Authored-by: Huaxin Gao <huaxing@us.ibm.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
Huaxin Gao 2020-04-14 13:29:44 +09:00 committed by HyukjinKwon
parent 31b907748d
commit 46be1e01e9
8 changed files with 48 additions and 11 deletions

View file

@ -231,7 +231,7 @@ the following case-insensitive options:
{% include_example jdbc_dataset r/RSparkSQLExample.R %} {% include_example jdbc_dataset r/RSparkSQLExample.R %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}

View file

@ -77,7 +77,7 @@ For a regular multi-line JSON file, set a named parameter `multiLine` to `TRUE`.
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}

View file

@ -127,7 +127,7 @@ visit the official Apache ORC/Parquet websites.
{% include_example manual_save_options_orc r/RSparkSQLExample.R %} {% include_example manual_save_options_orc r/RSparkSQLExample.R %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
CREATE TABLE users_with_options ( CREATE TABLE users_with_options (
@ -257,7 +257,7 @@ Bucketing and sorting are applicable only to persistent tables:
{% include_example write_sorting_and_bucketing python/sql/datasource.py %} {% include_example write_sorting_and_bucketing python/sql/datasource.py %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
@ -291,7 +291,7 @@ while partitioning can be used with both `save` and `saveAsTable` when using the
{% include_example write_partitioning python/sql/datasource.py %} {% include_example write_partitioning python/sql/datasource.py %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
@ -323,7 +323,7 @@ It is possible to use both partitioning and bucketing for a single table:
{% include_example write_partition_and_bucket python/sql/datasource.py %} {% include_example write_partition_and_bucket python/sql/datasource.py %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}

View file

@ -52,7 +52,7 @@ Using the data from the above example:
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
@ -242,7 +242,7 @@ refreshTable("my_table")
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
REFRESH TABLE my_table; REFRESH TABLE my_table;

View file

@ -205,7 +205,7 @@ refer it, e.g. `SELECT * FROM global_temp.view1`.
{% include_example global_temp_view python/sql/basic.py %} {% include_example global_temp_view python/sql/basic.py %}
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}

View file

@ -169,7 +169,7 @@ head(join(src, hint(records, "broadcast"), src$key == records$key))
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
{% highlight sql %} {% highlight sql %}
-- We accept BROADCAST, BROADCASTJOIN and MAPJOIN for broadcast hint -- We accept BROADCAST, BROADCASTJOIN and MAPJOIN for broadcast hint

View file

@ -631,7 +631,7 @@ from pyspark.sql.types import *
</table> </table>
</div> </div>
<div data-lang="sql" markdown="1"> <div data-lang="SQL" markdown="1">
The following table shows the type names as well as aliases used in Spark SQL parser for each data type. The following table shows the type names as well as aliases used in Spark SQL parser for each data type.

View file

@ -94,8 +94,45 @@ For example, a user-defined average for untyped DataFrames can look like:
<div data-lang="java" markdown="1"> <div data-lang="java" markdown="1">
{% include_example untyped_custom_aggregation java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java%} {% include_example untyped_custom_aggregation java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java%}
</div> </div>
<div data-lang="SQL" markdown="1">
{% highlight sql %}
-- Compile and place UDAF MyAverage in a JAR file called `MyAverage.jar` in /tmp.
CREATE FUNCTION myAverage AS 'MyAverage' USING JAR '/tmp/MyAverage.jar';
SHOW USER FUNCTIONS;
-- +------------------+
-- | function|
-- +------------------+
-- | default.myAverage|
-- +------------------+
CREATE TEMPORARY VIEW employees
USING org.apache.spark.sql.json
OPTIONS (
path "examples/src/main/resources/employees.json"
);
SELECT * FROM employees;
-- +-------+------+
-- | name|salary|
-- +-------+------+
-- |Michael| 3000|
-- | Andy| 4500|
-- | Justin| 3500|
-- | Berta| 4000|
-- +-------+------+
SELECT myAverage(salary) as average_salary FROM employees;
-- +--------------+
-- |average_salary|
-- +--------------+
-- | 3750.0|
-- +--------------+
{% endhighlight %}
</div>
</div> </div>
### Related Statements ### Related Statements
* [Scalar User Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html) * [Scalar User Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html)
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html) * [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)