[SPARK-31319][SQL][FOLLOW-UP] Add a SQL example for UDAF
### What changes were proposed in this pull request? Add a SQL example for UDAF ### Why are the changes needed? To make SQL Reference complete ### Does this PR introduce any user-facing change? Yes. Add the following page, also change ```Sql``` to ```SQL``` in the example tab for all the sql examples. <img width="1110" alt="Screen Shot 2020-04-13 at 6 09 24 PM" src="https://user-images.githubusercontent.com/13592258/79175240-06cd7400-7db2-11ea-8f3e-af71a591a64b.png"> ### How was this patch tested? Manually build and check Closes #28209 from huaxingao/udf_followup. Authored-by: Huaxin Gao <huaxing@us.ibm.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
parent
31b907748d
commit
46be1e01e9
|
@ -231,7 +231,7 @@ the following case-insensitive options:
|
|||
{% include_example jdbc_dataset r/RSparkSQLExample.R %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
|
|
@ -77,7 +77,7 @@ For a regular multi-line JSON file, set a named parameter `multiLine` to `TRUE`.
|
|||
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
|
|
@ -127,7 +127,7 @@ visit the official Apache ORC/Parquet websites.
|
|||
{% include_example manual_save_options_orc r/RSparkSQLExample.R %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
CREATE TABLE users_with_options (
|
||||
|
@ -257,7 +257,7 @@ Bucketing and sorting are applicable only to persistent tables:
|
|||
{% include_example write_sorting_and_bucketing python/sql/datasource.py %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
@ -291,7 +291,7 @@ while partitioning can be used with both `save` and `saveAsTable` when using the
|
|||
{% include_example write_partitioning python/sql/datasource.py %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
@ -323,7 +323,7 @@ It is possible to use both partitioning and bucketing for a single table:
|
|||
{% include_example write_partition_and_bucket python/sql/datasource.py %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
|
|
@ -52,7 +52,7 @@ Using the data from the above example:
|
|||
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
@ -242,7 +242,7 @@ refreshTable("my_table")
|
|||
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
REFRESH TABLE my_table;
|
||||
|
|
|
@ -205,7 +205,7 @@ refer it, e.g. `SELECT * FROM global_temp.view1`.
|
|||
{% include_example global_temp_view python/sql/basic.py %}
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
|
||||
|
|
|
@ -169,7 +169,7 @@ head(join(src, hint(records, "broadcast"), src$key == records$key))
|
|||
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
{% highlight sql %}
|
||||
-- We accept BROADCAST, BROADCASTJOIN and MAPJOIN for broadcast hint
|
||||
|
|
|
@ -631,7 +631,7 @@ from pyspark.sql.types import *
|
|||
</table>
|
||||
</div>
|
||||
|
||||
<div data-lang="sql" markdown="1">
|
||||
<div data-lang="SQL" markdown="1">
|
||||
|
||||
The following table shows the type names as well as aliases used in Spark SQL parser for each data type.
|
||||
|
||||
|
|
|
@ -94,8 +94,45 @@ For example, a user-defined average for untyped DataFrames can look like:
|
|||
<div data-lang="java" markdown="1">
|
||||
{% include_example untyped_custom_aggregation java/org/apache/spark/examples/sql/JavaUserDefinedUntypedAggregation.java%}
|
||||
</div>
|
||||
<div data-lang="SQL" markdown="1">
|
||||
{% highlight sql %}
|
||||
-- Compile and place UDAF MyAverage in a JAR file called `MyAverage.jar` in /tmp.
|
||||
CREATE FUNCTION myAverage AS 'MyAverage' USING JAR '/tmp/MyAverage.jar';
|
||||
|
||||
SHOW USER FUNCTIONS;
|
||||
-- +------------------+
|
||||
-- | function|
|
||||
-- +------------------+
|
||||
-- | default.myAverage|
|
||||
-- +------------------+
|
||||
|
||||
CREATE TEMPORARY VIEW employees
|
||||
USING org.apache.spark.sql.json
|
||||
OPTIONS (
|
||||
path "examples/src/main/resources/employees.json"
|
||||
);
|
||||
|
||||
SELECT * FROM employees;
|
||||
-- +-------+------+
|
||||
-- | name|salary|
|
||||
-- +-------+------+
|
||||
-- |Michael| 3000|
|
||||
-- | Andy| 4500|
|
||||
-- | Justin| 3500|
|
||||
-- | Berta| 4000|
|
||||
-- +-------+------+
|
||||
|
||||
SELECT myAverage(salary) as average_salary FROM employees;
|
||||
-- +--------------+
|
||||
-- |average_salary|
|
||||
-- +--------------+
|
||||
-- | 3750.0|
|
||||
-- +--------------+
|
||||
{% endhighlight %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
### Related Statements
|
||||
|
||||
* [Scalar User Defined Functions (UDFs)](sql-ref-functions-udf-scalar.html)
|
||||
* [Integration with Hive UDFs/UDAFs/UDTFs](sql-ref-functions-udf-hive.html)
|
||||
|
|
Loading…
Reference in a new issue