spark-instrumented-optimizer/dev/.rat-excludes
Thomas Graves b64688ebba [SPARK-29303][WEB UI] Add UI support for stage level scheduling
### What changes were proposed in this pull request?

This adds UI updates to support stage level scheduling and ResourceProfiles. 3 main things have been added. ResourceProfile id added to the Stage page, the Executors page now has an optional selectable column to show the ResourceProfile Id of each executor, and the Environment page now has a section with the ResourceProfile ids.  Along with this the rest api for environment page was updated to include the Resource profile information.

I debating on splitting the resource profile information into its own page but I wasn't sure it called for a completely separate page. Open to peoples thoughts on this.

Screen shots:
![Screen Shot 2020-04-01 at 3 07 46 PM](https://user-images.githubusercontent.com/4563792/78185169-469a7000-7430-11ea-8b0c-d9ede2d41df8.png)
![Screen Shot 2020-04-01 at 3 08 14 PM](https://user-images.githubusercontent.com/4563792/78185175-48fcca00-7430-11ea-8d1d-6b9333700f32.png)
![Screen Shot 2020-04-01 at 3 09 03 PM](https://user-images.githubusercontent.com/4563792/78185176-4a2df700-7430-11ea-92d9-73c382bb0f32.png)
![Screen Shot 2020-04-01 at 11 05 48 AM](https://user-images.githubusercontent.com/4563792/78185186-4dc17e00-7430-11ea-8962-f749dd47ea60.png)

### Why are the changes needed?

For user to be able to know what resource profile was used with which stage and executors. The resource profile information is also available so user debugging can see exactly what resources were requested with that profile.

### Does this PR introduce any user-facing change?

Yes, UI updates.

### How was this patch tested?

Unit tests and tested on yarn both active applications and with the history server.

Closes #28094 from tgravescs/SPARK-29303-pr.

Lead-authored-by: Thomas Graves <tgraves@nvidia.com>
Co-authored-by: Thomas Graves <tgraves@apache.org>
Signed-off-by: Thomas Graves <tgraves@apache.org>
2020-05-21 13:11:35 -05:00

128 lines
2.1 KiB
Plaintext

target
cache
.gitignore
.gitattributes
.project
.classpath
.mima-excludes
.generated-mima-excludes
.generated-mima-class-excludes
.generated-mima-member-excludes
.rat-excludes
.*md
derby.log
licenses/*
licenses-binary/*
LICENSE
NOTICE
TAGS
RELEASE
control
docs
slaves
spark-env.cmd
bootstrap.bundle.min.js
bootstrap.bundle.min.js.map
bootstrap.min.css
bootstrap.min.css.map
jquery-3.4.1.min.js
d3.min.js
dagre-d3.min.js
graphlib-dot.min.js
sorttable.js
vis-timeline-graph2d.min.js
vis-timeline-graph2d.min.js.map
vis-timeline-graph2d.min.css
vis-timeline-graph2d.min.css.map
dataTables.bootstrap4.1.10.20.min.css
dataTables.bootstrap4.1.10.20.min.js
dataTables.rowsGroup.js
jquery.blockUI.min.js
jquery.cookies.2.2.0.min.js
jquery.dataTables.1.10.20.min.css
jquery.dataTables.1.10.20.min.js
jquery.mustache.js
jsonFormatter.min.css
jsonFormatter.min.js
.*avsc
.*txt
.*json
.*data
.*log
pyspark-coverage-site/
cloudpickle.py
heapq3.py
join.py
SparkExprTyper.scala
SparkILoop.scala
SparkILoopInit.scala
SparkIMain.scala
SparkImports.scala
SparkJLineCompletion.scala
SparkJLineReader.scala
SparkMemberHandlers.scala
SparkReplReporter.scala
sbt
sbt-launch-lib.bash
plugins.sbt
work
.*\.q
.*\.qv
golden
test.out/*
.*iml
service.properties
db.lck
build/*
dist/*
.*out
.*ipr
.*iws
logs
.*scalastyle-output.xml
.*dependency-reduced-pom.xml
known_translations
json_expectation
app-20180109111548-0000
app-20161115172038-0000
app-20161116163331-0000
application_1516285256255_0012
application_1553914137147_0018
application_1555004656427_0144
stat
local-1422981759269
local-1422981780767
local-1425081759269
local-1426533911241
local-1426633911242
local-1430917381534
local-1430917381535_1
local-1430917381535_2
DESCRIPTION
NAMESPACE
test_support/*
.*Rd
help/*
html/*
INDEX
.lintr
gen-java.*
.*avpr
.*parquet
spark-deps-.*
.*csv
.*tsv
.*\.sql
.Rbuildignore
META-INF/*
spark-warehouse
structured-streaming/*
kafka-source-initial-offset-version-2.1.0.bin
kafka-source-initial-offset-future-version.bin
announce.tmpl
vote.tmpl
SessionManager.java
SessionHandler.java
GangliaReporter.java
application_1578436911597_0052