diff --git a/docs/monitoring.md b/docs/monitoring.md index 5fdf3083b4..31fc16025c 100644 --- a/docs/monitoring.md +++ b/docs/monitoring.md @@ -758,7 +758,7 @@ The JSON end point is exposed at: `/applications/[app-id]/executors`, and the Pr The Prometheus endpoint is experimental and conditional to a configuration parameter: `spark.ui.prometheus.enabled=true` (the default is `false`). In addition, aggregated per-stage peak values of the executor memory metrics are written to the event log if `spark.eventLog.logStageExecutorMetrics` is true. -Executor memory metrics are also exposed via the Spark metrics system based on the Dropwizard metrics library. +Executor memory metrics are also exposed via the Spark metrics system based on the [Dropwizard metrics library](http://metrics.dropwizard.io/4.1.1). A list of the available metrics, with a short description: @@ -962,7 +962,7 @@ keep the paths consistent in both modes. # Metrics Spark has a configurable metrics system based on the -[Dropwizard Metrics Library](http://metrics.dropwizard.io/). +[Dropwizard Metrics Library](http://metrics.dropwizard.io/4.1.1). This allows users to report Spark metrics to a variety of sinks including HTTP, JMX, and CSV files. The metrics are generated by sources embedded in the Spark code base. They provide instrumentation for specific activities and Spark components. @@ -1056,7 +1056,7 @@ activates the JVM source: ## List of available metrics providers Metrics used by Spark are of multiple types: gauge, counter, histogram, meter and timer, -see [Dropwizard library documentation for details](https://metrics.dropwizard.io/3.1.0/getting-started/). +see [Dropwizard library documentation for details](https://metrics.dropwizard.io/4.1.1/getting-started.html). The following list of components and metrics reports the name and some details about the available metrics, grouped per component instance and source namespace. The most common time of metrics used in Spark instrumentation are gauges and counters. @@ -1284,7 +1284,7 @@ Notes: `spark.metrics.staticSources.enabled` (default is true) - This source is available for driver and executor instances and is also available for other instances. - This source provides information on JVM metrics using the - [Dropwizard/Codahale Metric Sets for JVM instrumentation](https://metrics.dropwizard.io/3.1.0/manual/jvm/) + [Dropwizard/Codahale Metric Sets for JVM instrumentation](https://metrics.dropwizard.io/4.1.1/manual/jvm.html) and in particular the metric sets BufferPoolMetricSet, GarbageCollectorMetricSet and MemoryUsageGaugeSet. ### Component instance = applicationMaster diff --git a/pom.xml b/pom.xml index e414835d8d..23de5697f6 100644 --- a/pom.xml +++ b/pom.xml @@ -145,6 +145,10 @@ 0.9.52.4.02.0.8 + 4.1.11.8.2hadoop2