ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-33840][DOCS] Add spark.sql.files.minPartitionNum to performence tuning doc

Browse source 
### What changes were proposed in this pull request?

Add `spark.sql.files.minPartitionNum` and it's description to sql-performence-tuning.md.

### Why are the changes needed?

Help user to find it.

### Does this PR introduce _any_ user-facing change?

Yes, it's the doc.

### How was this patch tested?

Pass CI.

Closes #30838 from ulysses-you/SPARK-33840.

Authored-by: ulysses-you <ulyssesyou18@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
ulysses-you 2020-12-18 20:27:16 +09:00 committed by HyukjinKwon
parent 0603913c66
commit bc46d273e0
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/sql-performance-tuning.md
View file

@ -85,6 +85,16 @@ that these options will be deprecated in future release as more optimizations ar
</td>
<td>2.0.0</td>
</tr>
<tr>
<td><code>spark.sql.files.minPartitionNum</code></td>
<td>Default Parallelism</td>
<td>
The suggested (not guaranteed) minimum number of split file partitions. If not set, the default
value is `spark.default.parallelism`. This configuration is effective only when using file-based
sources such as Parquet, JSON and ORC.
</td>
<td>3.1.0</td>
</tr>
<tr>
<td><code>spark.sql.broadcastTimeout</code></td>
<td>300</td>