spark-instrumented-optimizer/sql/core
Wenchen Fan 34414acfa3 [SPARK-31706][SQL] add back the support of streaming update mode
### What changes were proposed in this pull request?

This PR adds a private `WriteBuilder` mixin trait: `SupportsStreamingUpdate`, so that the builtin v2 streaming sinks can still support the update mode.

Note: it's private because we don't have a proper design yet. I didn't take the proposal in https://github.com/apache/spark/pull/23702#discussion_r258593059 because we may want something more general, like updating by an expression `key1 = key2 + 10`.

### Why are the changes needed?

In Spark 2.4, all builtin v2 streaming sinks support all streaming output modes, and v2 sinks are enabled by default, see https://issues.apache.org/jira/browse/SPARK-22911

It's too risky for 3.0 to go back to v1 sinks, so I propose to add a private trait to fix builtin v2 sinks, to keep backward compatibility.

### Does this PR introduce _any_ user-facing change?

Yes, now all the builtin v2 streaming sinks support all streaming output modes, which is the same as 2.4

### How was this patch tested?

existing tests.

Closes #28523 from cloud-fan/update.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2020-05-20 03:45:13 +00:00
..
benchmarks [SPARK-31630][SQL] Fix perf regression by skipping timestamps rebasing after some threshold 2020-05-05 14:11:53 +00:00
src [SPARK-31706][SQL] add back the support of streaming update mode 2020-05-20 03:45:13 +00:00
v1.2/src [SPARK-31489][SQL] Fix pushing down filters with java.time.LocalDate values in ORC 2020-04-26 15:49:00 -07:00
v2.3/src [SPARK-31489][SQL] Fix pushing down filters with java.time.LocalDate values in ORC 2020-04-26 15:49:00 -07:00
pom.xml [SPARK-31272][SQL] Support DB2 Kerberos login in JDBC connector 2020-04-22 17:10:30 -07:00