ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-32409][DOC] Document dependency between spark.metrics.staticSources.enabled and JVMSource registration

Browse source 
### What changes were proposed in this pull request?
Document the dependency between the config `spark.metrics.staticSources.enabled` and JVMSource registration.

### Why are the changes needed?

This PT just documents the dependency between config `spark.metrics.staticSources.enabled` and JVM source registration.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?
Manually tested.

Closes #29203 from LucaCanali/bugJVMMetricsRegistration.

Authored-by: Luca Canali <luca.canali@cern.ch>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
Luca Canali 2020-08-10 09:32:01 -07:00 committed by Dongjoon Hyun
parent b421bf0196
commit 99f50c6286
1 changed files with 6 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/monitoring.md
View file

@ -1082,7 +1082,7 @@ This is the component with the largest amount of instrumented metrics
- memory.remainingOnHeapMem_MB
- namespace=HiveExternalCatalog
- **note:**: these metrics are conditional to a configuration parameter:
- **note:** these metrics are conditional to a configuration parameter:
`spark.metrics.staticSources.enabled` (default is true)
- fileCacheHits.count
- filesDiscovered.count
@ -1091,7 +1091,7 @@ This is the component with the largest amount of instrumented metrics
- partitionsFetched.count
- namespace=CodeGenerator
- **note:**: these metrics are conditional to a configuration parameter:
- **note:** these metrics are conditional to a configuration parameter:
`spark.metrics.staticSources.enabled` (default is true)
- compilationTime (histogram)
- generatedClassSize (histogram)
@ -1254,7 +1254,7 @@ when running in local mode.
- shuffle-server.usedHeapMemory
- namespace=HiveExternalCatalog
- **note:**: these metrics are conditional to a configuration parameter:
- **note:** these metrics are conditional to a configuration parameter:
`spark.metrics.staticSources.enabled` (default is true)
- fileCacheHits.count
- filesDiscovered.count
@ -1263,7 +1263,7 @@ when running in local mode.
- partitionsFetched.count
- namespace=CodeGenerator
- **note:**: these metrics are conditional to a configuration parameter:
- **note:** these metrics are conditional to a configuration parameter:
`spark.metrics.staticSources.enabled` (default is true)
- compilationTime (histogram)
- generatedClassSize (histogram)
@ -1280,6 +1280,8 @@ when running in local mode.
Notes:
- Activate this source by setting the relevant `metrics.properties` file entry or the
configuration parameter:`spark.metrics.conf.*.source.jvm.class=org.apache.spark.metrics.source.JvmSource`
- These metrics are conditional to a configuration parameter:
`spark.metrics.staticSources.enabled` (default is true)
- This source is available for driver and executor instances and is also available for other instances.
- This source provides information on JVM metrics using the
[Dropwizard/Codahale Metric Sets for JVM instrumentation](https://metrics.dropwizard.io/3.1.0/manual/jvm/)