f6e6d1157a
In stage-level resource scheduling, the allocated 3rd party resources can be obtained in TaskContext using resources() interface, however there is no API to get how many cpus are allocated for the task. Will add a cpus() interface to TaskContext to complement resources(). Althrough the task cpu requests can be got from profile, it's more convenient to get it inside the task code without the need to pass profile from driver side to the executor side. ### What changes were proposed in this pull request? Add cpus() interface in TaskContext and modify relevant code. ### Why are the changes needed? TaskContext has resources() to get 3rd party resources allocated. the is no API to get CPU allocated for the task. ### Does this PR introduce _any_ user-facing change? Add cpus() interface for TaskContext ### How was this patch tested? Unit tests Closes #33385 from xwu99/taskcontext-cpus. Lead-authored-by: Wu, Xiaochang <xiaochang.wu@intel.com> Co-authored-by: Xiaochang Wu <xiaochang.wu@intel.com> Signed-off-by: Mridul Muralidharan <mridul<at>gmail.com> |
||
|---|---|---|
| .. | ||
| build.properties | ||
| MimaBuild.scala | ||
| MimaExcludes.scala | ||
| plugins.sbt | ||
| SparkBuild.scala | ||