Liang-Chi Hsieh 6cd5cf5722 [SPARK-35215][SQL] Update custom metric per certain rows and at the end of the task ... ### What changes were proposed in this pull request? This patch changes custom metric updating to update per certain rows (currently 100), instead of per row. ### Why are the changes needed? Based on previous discussion https://github.com/apache/spark/pull/31451#discussion_r605413557, we should only update custom metrics per certain (e.g. 100) rows and also at the end of the task. Updating per row doesn't make too much benefit. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Existing unit test. Closes #32330 from viirya/metric-update. Authored-by: Liang-Chi Hsieh <viirya@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>		2021-05-06 13:21:08 +00:00
..
benchmarks	[SPARK-35235][SQL][TEST] Add row-based hash map into aggregate benchmark	2021-04-27 06:53:42 +00:00
src	[SPARK-35215][SQL] Update custom metric per certain rows and at the end of the task	2021-05-06 13:21:08 +00:00
pom.xml	[SPARK-33662][BUILD] Setting version to 3.2.0-SNAPSHOT	2020-12-04 14:10:42 -08:00