ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-33397][YARN][DOC] Fix generating md to html for available-patterns-for-shs-custom-executor-log-url

Browse source 
### What changes were proposed in this pull request?

1. replace `{{}}`  with `{{}}`
2. using `<code></code>` in td-tag

### Why are the changes needed?

to fix this.
![image](https://user-images.githubusercontent.com/8326978/98544155-8c74bc00-22ce-11eb-8889-8dacb726b762.png)

### Does this PR introduce _any_ user-facing change?

yes, you will see the correct online doc with this change

![image](https://user-images.githubusercontent.com/8326978/98545256-2e48d880-22d0-11eb-9dd9-b8cae3df8659.png)

### How was this patch tested?

shown as the above pic via jekyll serve.

Closes #30298 from yaooqinn/SPARK-33397.

Authored-by: Kent Yao <yaooqinn@hotmail.com>
Signed-off-by: Takeshi Yamamuro <yamamuro@apache.org>
This commit is contained in:
Kent Yao 2020-11-10 10:15:55 +09:00 committed by Takeshi Yamamuro
parent 090962cd42
commit 036c11b0d4
1 changed files with 15 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

30
docs/running-on-yarn.md
View file

@ -584,48 +584,48 @@ To use a custom metrics.properties for the application master and executors, upd
<table class="table"> <table class="table">
<tr><th>Pattern</th><th>Meaning</th></tr> <tr><th>Pattern</th><th>Meaning</th></tr>
<tr> <tr>
<td>{{HTTP_SCHEME}}</td> <td>&#123;&#123;HTTP_SCHEME&#125;&#125;</td>
<td>`http://` or `https://` according to YARN HTTP policy. (Configured via `yarn.http.policy`)</td> <td><code>http://</code> or <code>https://</code> according to YARN HTTP policy. (Configured via <code>yarn.http.policy</code>)</td>
</tr> </tr>
<tr> <tr>
<td>{{NM_HOST}}</td> <td>&#123;&#123;NM_HOST&#125;&#125;</td>
<td>The "host" of node where container was run.</td> <td>The "host" of node where container was run.</td>
</tr> </tr>
<tr> <tr>
<td>{{NM_PORT}}</td> <td>&#123;&#123;NM_PORT&#125;&#125;</td>
<td>The "port" of node manager where container was run.</td> <td>The "port" of node manager where container was run.</td>
</tr> </tr>
<tr> <tr>
<td>{{NM_HTTP_PORT}}</td> <td>&#123;&#123;NM_HTTP_PORT&#125;&#125;</td>
<td>The "port" of node manager's http server where container was run.</td> <td>The "port" of node manager's http server where container was run.</td>
</tr> </tr>
<tr> <tr>
<td>{{NM_HTTP_ADDRESS}}</td> <td>&#123;&#123;NM_HTTP_ADDRESS&#125;&#125;</td>
<td>Http URI of the node on which the container is allocated.</td> <td>Http URI of the node on which the container is allocated.</td>
</tr> </tr>
<tr> <tr>
<td>{{CLUSTER_ID}}</td> <td>&#123;&#123;CLUSTER_ID&#125;&#125;</td>
<td>The cluster ID of Resource Manager. (Configured via `yarn.resourcemanager.cluster-id`)</td> <td>The cluster ID of Resource Manager. (Configured via <code>yarn.resourcemanager.cluster-id</code>)</td>
</tr> </tr>
<tr> <tr>
<td>{{CONTAINER_ID}}</td> <td>&#123;&#123;CONTAINER_ID&#125;&#125;</td>
<td>The ID of container.</td> <td>The ID of container.</td>
</tr> </tr>
<tr> <tr>
<td>{{USER}}</td> <td>&#123;&#123;USER&#125;&#125;</td>
<td>'SPARK_USER' on system environment.</td> <td><code>SPARK_USER</code> on system environment.</td>
</tr> </tr>
<tr> <tr>
<td>{{FILE_NAME}}</td> <td>&#123;&#123;FILE_NAME&#125;&#125;</td>
<td>`stdout`, `stderr`.</td> <td><code>stdout</code>, <code>stderr</code>.</td>
</tr> </tr>
</table> </table>
For example, suppose you would like to point log url link to Job History Server directly instead of let NodeManager http server redirects it, you can configure `spark.history.custom.executor.log.url` as below: For example, suppose you would like to point log url link to Job History Server directly instead of let NodeManager http server redirects it, you can configure `spark.history.custom.executor.log.url` as below:
`{{HTTP_SCHEME}}<JHS_HOST>:<JHS_PORT>/jobhistory/logs/{{NM_HOST}}:{{NM_PORT}}/{{CONTAINER_ID}}/{{CONTAINER_ID}}/{{USER}}/{{FILE_NAME}}?start=-4096` <code>&#123;&#123;HTTP_SCHEME&#125;&#125;&lt;JHS_HOST&gt;:&lt;JHS_PORT&gt;/jobhistory/logs/&#123;&#123;NM_HOST&#125;&#125;:&#123;&#123;NM_PORT&#125;&#125;/&#123;&#123;CONTAINER_ID&#125;&#125;/&#123;&#123;CONTAINER_ID&#125;&#125;/&#123;&#123;USER&#125;&#125;/&#123;&#123;FILE_NAME&#125;&#125;?start=-4096</code>
NOTE: you need to replace `<JHS_POST>` and `<JHS_PORT>` with actual value. NOTE: you need to replace `<JHS_POST>` and `<JHS_PORT>` with actual value.
# Resource Allocation and Configuration Overview # Resource Allocation and Configuration Overview