ODIn/Website
9
0
Fork 1
Code Pull requests Releases Wiki Activity

Slides

Browse source
This commit is contained in:
Oliver Kennedy 2019-10-22 10:54:39 -04:00
parent 9e35fcd52c
commit 6c60a38999
Signed by: okennedy
GPG key ID: 3E5F9B3ABD3FDB60
2 changed files with 1 additions and 469 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

463
src/teaching/cse-662/2019fa/slide/2019-10-15-Legorithmics.erb
View file

@ -1,463 +0,0 @@
---
template: templates/cse662_2019_slides.erb
title: Legorithmics
date: October 15
---
<section>
<section>
<h2>Legorithmics</h2>
<h4>October 23, 2017</h4>
</section>
<section>
<svg width=700 height=450>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper6.png"
x="50" y="0"
width="446" height="183"
border="1"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper3.png"
x="250" y="60"
width="424" height="183"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper5.png"
x="0" y="160"
width="422" height="190"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper4.png"
x="200" y="220"
width="473" height="229"
/>
</svg>
<p class="fragment"><b>Same great algorithms, <br/> awesome new hardware flavor</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>Hardware adaptation uses standard transformations:
<ul>
<li class="fragment"><b>Batching</b>: Prefetch blocks of data to avoid random seeks.</li>
<li class="fragment"><b>Partitioning</b>: Group related blocks of data to minimize cross-partition work.</li>
<li class="fragment"><b>Reordering</b>: Cluster data accesses for better cache locality.</li>
</ul>
</p>
<p class="fragment">The hard part is picking <b>where to apply the transformations</b> and selecting <b>values for the transformation's parameters</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>Key Insights:
<ul>
<li>The basic algorithms haven't changed since the 70s.</li>
<li>Hardware changes very slowly</li>
<li class="fragment">You have as much time as you need to design a hardware-specific algorithm</li>
</ul>
</p>
<p class="fragment"><b>Automate the search!</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>What do we need?</p>
<ul>
<li class="fragment">A way to describe the algorithm.</li>
<li class="fragment">A way to describe the hardware.</li>
<li class="fragment">A cost model.</li>
<li class="fragment">Transformation rules.</li>
<li class="fragment">A (possibly exp-time) optimizer</li>
</ul>
</section>
</section>
<section>
<section>
<h3>Typesystems</h3>
<p>A collection of rules that assign a property called a <i>type</i> to the parts of a computer program: variables, expressions, etc...<br/><attribution>[Wikipedia]</attribution></p>
</section>
<section>
<h3>Typesystems</h3>
<p>A typesystem allows you to:
<ul>
<li class="fragment">Define interfaces between different parts of a program.</li>
<li class="fragment">Check that these parts have been connected consistently.</li>
<li class="fragment"><span class="fragment highlight-blue">Define global properties in terms of local properties.</span></li>
</ul>
</p>
</section>
<section>
<h3>Typesystems</h3>
<br/>
<p><b>A type</b><br/>
($\tau := D\;|\;[\tau]\;|<\tau,\tau>\;|\;\tau\rightarrow\tau$)
</p>
<br/>
<p><b>A set of inference rules</b><br/>
($\frac{e\;:\;\tau}{[e]\;:\;[\tau]}$)
</p>
<br/>
<p class="fragment">These example types are part of the monad algebra</p>
</section>
<section>
<h3>Types</h3>
<table>
<tr><th>Type</th><th>Meaning</th></tr>
<tr><td>$D$</td><td>Primitive Type (int, float, etc...)</td></tr>
<tr><td>$[\tau]$</td><td>An array of elements with type $\tau$</td></tr>
<tr><td>$<\tau_1,\tau_2>$</td><td>A pair of elements of types $\tau_1$ and $\tau_2$.</td></tr>
<tr><td>$\tau_1\rightarrow\tau_2$</td><td>A function with one argument of type $\tau_1$ and one return value of type $\tau_2$.</td></tr>
</table>
</section>
<section>
<h3>Type Examples</h3>
<br/><div class="fragment">$[ < int,float > ]$</div>
<br/><div class="fragment">$[int] \rightarrow float$</div>
<br/><div class="fragment">$ < [int], [int] >\; \rightarrow [ < int,int > ]$</div>
</section>
<section>
<h3>Inference Rules</h3>
<br/>
<p>Defined over a language of expressions like $f(a, b)$.</p>
$$\frac{a : \tau_a\;\;\;b : \tau_b}{f(a,b):\tau_f(\tau_a, \tau_b)}$$
<p class="fragment">If expression $a$ has type $\tau_a$ and expression $b$ has type $\tau_b$...</p>
<p class="fragment">...then expression $f(a, b)$ has type $\tau_f(\tau_a, \tau_b)$.</p>
</section>
<section>
<h3>Inference Examples</h3>
<br/><div class="fragment">
$\frac{e: \tau}{[e] : [\tau]}$
</div>
<br/><div class="fragment">
$\frac{c: Bool\;\;\;e_1:\tau\;\;\;e_2:\tau}{(\textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2)\;:\; \tau}$
</div>
<br/><div class="fragment">
$\frac{e: < \tau_1, \tau_2 >\;\;\;i \in \{1,2\}}{e.i\; :\; \tau_i}$
</div>
</section>
</section>
<section>
<section>
<h3>Monad Algebra</h3>
<p>A primitive language for describing data processing.</p>
<table>
<tr><th>Operator</th><th>Meaning</th></tr>
<tr><td>$\lambda x.e$</td><td>Define a function with body $e$ that uses variable $x$.</td></tr>
<tr><td>$e_1\;e_2$</td><td>Apply the function defined by $e_1$ to the value obtained from $e_2$.</td></tr>
<tr><td>$ \textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2$</td><td>If $c$ is true then evaluate $e_1$, and otherwise evaluate $e_2$.</td></tr>
</table>
</section>
<section>
<h3>Monad Algebra</h3>
<table>
<tr><th>Operator</th><th>Meaning</th></tr>
<tr><td>$ < e_1, e_2 >$</td><td>Construct a tuple from $e_1$ and $e_2$.</td></tr>
<tr><td>$ e.i $</td><td>Extract attribute $i$ from the tuple $e$.</td></tr>
<tr><td>$ [e] $</td><td>Construct a single-element array from e.</td></tr>
<tr><td>$ [] $</td><td>Construct an empty array.</td></tr>
<tr><td>$ e_1 \sqcup e_2 $</td><td>Concatenate the arrays $e_1$ and $e_2$.</td></tr>
</table>
</section>
<section>
$$\textbf{flatMap}(f : \tau_1 \rightarrow [\tau_2])(e : [\tau_1])$$
<p class="fragment">Apply function $f$ to every element of array $e$. Concatenate all of the arrays returned by $f$.</p>
$$\textbf{foldL}(c : \tau_2, f : < \tau_2, \tau_1 >)(e : [\tau_1])$$
<p class="fragment">Apply function $f$ to every element of array $e$, with each invocation passing its return value to the next call (e.g., aggregation)<p>
$$\textbf{for}(xB : [\tau_1] [k] \leftarrow e_{in} : [\tau_1])(e_{loop} : [\tau_2])$$
<p class="fragment">Extract blocks of size $k$ from $e_{in}$. For each block compute a <b>flatMap</b> using expression $e_{loop}$.<p>
</section>
<section>
<h3>Example - Average</h3>
<div style="text-align: left">
<span class="fragment" data-fragment-index=6>$(\lambda tot.(tot.1 / tot.2))$</span><br/>
&nbsp;&nbsp;&nbsp;&nbsp;
<span class="fragment" data-fragment-index=1>$(\textbf{foldL}($
<span class="fragment" data-fragment-index=5>$< 0, 0 >,$</span>
<span class="fragment" data-fragment-index=2>$(\lambda < a, x >.<$
<span class="fragment" data-fragment-index=3>$a.1 + x$</span>
$,$
<span class="fragment" data-fragment-index=4>$a.2 + 1$</span>
$ > $</span>
$))$</span>
</div>
<p class="fragment" data-fragment-index=1>Fold implements aggregation</p>
<p class="fragment" data-fragment-index=2>Fold takes a 'previous' $a$ and a 'current' $x$</p>
<p class="fragment" data-fragment-index=3>We need a sum<span class="fragment" data-fragment-index=4>, and a count</span></p>
<p class="fragment" data-fragment-index=5>Initial sum and count are both 0</p>
<p class="fragment" data-fragment-index=6>Postprocess with division ($\lambda$ creates a variable $tot$)</p>
</section>
</section>
<section>
<section>
<h3>Cost Estimation</h3>
<p>We need...
<ul>
<li class="fragment">Cardinality estimation</li>
<li class="fragment">A model of IO</li>
<li class="fragment">IO Cost relative to cardinality</li>
</ul></p>
</section>
<section>
<h3>Cardinality Estimation</h3>
<p><b>Basic Approach:</b> Define a second <i>type</i> for tracking data sizes</p>
$$\alpha\;:=\;[\alpha]^x\;|\;< \alpha_1, \alpha_2 >|\;c$$
<br/>
<div class="fragment">
<p>e.g., $[ < 1, [1]^y > ]^x$ corresponds to:
<ul>
<li>an array with $x$ elements, where each element consists of...</li>
<li> a value of fixed-size 1, and...</li>
<li> an array of $y$ fixed-size values.</li>
</ul></p>
</div>
</section>
<section>
<h3>Cardinality Estimation</h3>
<ul>
<li>$size(c) = c$</li>
<li>$size([\alpha]^x) = x \cdot size(\alpha)$</li>
<li>$size( < \alpha_1, \alpha_2 >) = size(\alpha_1) + size(\alpha_2)$</li>
</ul>
</section>
<section>
<h3>Cardinality Estimation</h3>
<p>$R(\Gamma, e)$ computes the cardinality type for $e$</p>
<p>$\Gamma : x \rightarrow \alpha$ is a context / scope</p>
</section>
<section>
<h3>For Loops</h3>
<br/>
$$R\left(\Gamma, \textbf{for}(x [k] \leftarrow e_1)\; e_2\right) := $$
<p>
<span class="fragment" data-fragment-index=2>$\frac{cardinality(R(\Gamma, e_1))}{k}\cdot$</span>
<span class="fragment" data-fragment-index=1>$R(\Gamma', e_2)$</span>
</p>
<br/>
<div class="fragment" data-fragment-index=3>$\Gamma' := \Gamma \cup \{x \mapsto [sizeofElement(R(\Gamma, e_1))]^k\}$</div>
<br/>
<p class="fragment" data-fragment-index=1>The cardinality is based on that of $e_2$</p>
<p class="fragment" data-fragment-index=2>Repeated once for every time through the loop</p>
<p class="fragment" data-fragment-index=3>And $e_2$ is evaluated in the context of a $k$ element array.</p>
</section>
<section>
<h3>If Then Else</h3>
<br/>
$$R\left(\Gamma, \textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2\right) := $$
<p>
$max(R\left(\Gamma, e_1\right), R\left(\Gamma, e_2\right))$
</p>
<p class="fragment">Pessimistic assumption of biggest possible size. <br/> Avoids needing to estimate $p(c = true)$.</p>
</section>
</section>
<section>
<section>
<h3>Example: Block-Nested-Loop Join</h3>
<div style="text-align: left;">
<div style="float: right"># Loop over blocks in outer rel.</div>
<div style="margin-left: 0px">$\textbf{for}( xB [k_1] \leftarrow R )$</div>
<div style="float: right"># Loop over blocks in inner rel.</div>
<div style="margin-left: 30px">$\textbf{for}( yB [k_2] \leftarrow S )$</div>
<div style="float: right"># Loop over elems in outer block.</div>
<div style="margin-left: 60px">$\textbf{for}( x \leftarrow xB )$</div>
<div style="float: right"># Loop over elems in inner block.</div>
<div style="margin-left: 90px">$\textbf{for}( y \leftarrow yB )$</div>
<div style="float: right"># Join test.</div>
<div style="margin-left:120px">$\textbf{if}\;joinCond(x, y)$</div>
<div style="float: right"># Add pair if success.</div>
<div style="margin-left:150px">$\textbf{then}\;[< x, y >]$</div>
<div style="float: right"># Add nothing if not.</div>
<div style="margin-left:150px">$\textbf{else}\;[]$</div>
</div>
</section>
<section>
<table>
<tr><th>Expression</th><th>Context</th><th>Result Size</th></tr>
<tr><td>$\textbf{for}( xB [k_1] \leftarrow R )$</td>
<td class="fragment"
data-fragment-index=1
style="text-align: left">$\Gamma_1 = R \mapsto [1]^x, S \mapsto [1]^y$</td>
<td class="fragment"
data-fragment-index=14
>$[ < 1, 1 > ]^{\frac{x}{k_1} \cdot \frac{y}{k_2} \cdot k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( yB [k_2] \leftarrow S )$</td>
<td class="fragment"
data-fragment-index=2
style="text-align: left">$\Gamma_2 = \Gamma_1 \cup xB \mapsto [1]^{k_1}$</td>
<td class="fragment"
data-fragment-index=13
>$[ < 1, 1 > ]^{\frac{y}{k_2} \cdot k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( x \leftarrow xB )$</td>
<td class="fragment"
data-fragment-index=3
style="text-align: left">$\Gamma_3 = \Gamma_2 \cup yB \mapsto [1]^{k_2}$</td>
<td class="fragment"
data-fragment-index=12
>$[ < 1, 1 > ]^{k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( y \leftarrow yB )$</td>
<td class="fragment"
data-fragment-index=4
style="text-align: left">$\Gamma_4 = \Gamma_3 \cup x \mapsto 1$</td>
<td class="fragment"
data-fragment-index=11
>$[ < 1, 1 > ]^{k_2}$</td></tr>
<tr><td>$\textbf{if}\;joinCond(x, y)$</td>
<td class="fragment"
data-fragment-index=5
style="text-align: left">$\Gamma_5 = \Gamma_4 \cup y \mapsto 1$</td>
<td class="fragment"
data-fragment-index=10
>$[ < 1, 1 > ]^1$</td></tr>
<tr><td>$\textbf{then}\;[< x, y >]$</td>
<td class="fragment"
data-fragment-index=6
style="text-align: left">$\Gamma_5$</td>
<td class="fragment"
data-fragment-index=9
>$[ < 1, 1 > ]^1$</td></tr>
<tr><td>$\textbf{else}\;[]$</td>
<td class="fragment"
data-fragment-index=6
style="text-align: left">$\Gamma_5$</td>
<td class="fragment"
data-fragment-index=8
>$0$</td></tr>
</table>
</section>
</section>
<section>
<section>
<h3>IO Model</h3>
<p>IO Costs have 2 components:
<small><ul>
<li>$InitCom$: The cost of initializing a connection (e.g., seek time).</li>
<li>$UnitTr$: The cost of transferring one unit of data.</li>
</ul></small></p>
<p>Costs are defined for every pair of memory hierarchy levels:
<small><ul>
<li>$UnitTr(HDD \rightarrow RAM)$ is the cost of reading from HDD into Ram.</li>
<li>$InitCom(RAM \rightarrow HDD)$ is the cost of seeking to a write from Ram onto a HDD.</li>
</small></ul></p>
</section>
<section>
<table>
<tr><th>Expression</th><th>Result Size</th><th>HDD&nbsp;to&nbsp;RAM</th><th>RAM&nbsp;to&nbsp;HDD</th></tr>
<tr><td><small>$\textbf{for}( xB [k_1] \leftarrow R )$</small></td>
<td><small>$[ < 1, 1 > ]^{\frac{x}{k_1} \cdot \frac{y}{k_2} \cdot k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=7>$x+\frac{x}{k_1}y$</td>
<td class="fragment" data-fragment-index=4>$2xy$</td>
</tr>
<tr><td><small>$\textbf{for}( yB [k_2] \leftarrow S )$</small></td>
<td><small>$[ < 1, 1 > ]^{\frac{y}{k_2} \cdot k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=6>$y$</td>
<td class="fragment" data-fragment-index=4>$2k_1y$</td>
</tr>
<tr><td><small>$\textbf{for}( x \leftarrow xB )$</small></td>
<td><small>$[ < 1, 1 > ]^{k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=5>$0$</td>
<td class="fragment" data-fragment-index=4>$2k_1k_2$</td>
</tr>
<tr><td><small>$\textbf{for}( y \leftarrow yB )$</small></td>
<td><small>$[ < 1, 1 > ]^{k_2}$</small></td>
<td class="fragment" data-fragment-index=5>$0$</td>
<td class="fragment" data-fragment-index=3>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{if}\;joinCond(x, y)$</small></td>
<td><small>$[ < 1, 1 > ]^1$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=3>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{then}\;[< x, y >]$</small></td>
<td><small>$[ < 1, 1 > ]^1$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=2>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{else}\;[]$</small></td>
<td><small>$0$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=1>$0$</td>
</tr>
</table>
<p>HDD: $R, S, Result$<span style="margin-left: 100px">&nbsp;</span>RAM: $x, xB, y, yB$</p>
</section>
</section>
<section>
<section>
<h3>Rewrite Rules</h3>
<h4>Batching</h4>
$$for(x [1] \leftarrow R)\; e \Rightarrow for(xB [k] \leftarrow R)\; for(x [1] \leftarrow xB)\; e$$
<h4 style="margin-top: 30px">Reordering Iterators</h4>
$$for(x_1 [k_1] \leftarrow R_1)\;for(x_2 [k_2] \leftarrow R_2) \Rightarrow$$
$$for(x_2 [k_2] \leftarrow R_2)\;for(x_1 [k_1] \leftarrow R_1)$$
<h4 style="margin-top: 30px">Size-Dependent, Commutative Functions</h4>
$$f \Rightarrow (\lambda< x_1, x_2>.f(\textbf{if}\;|x_1|\leq |x_2|\;\textbf{then}< x_1, x_2 >\;\textbf{else}\;< x_2, x_1 >))$$
$$f \Rightarrow (\lambda< x_1, x_2>.f(\textbf{if}\;|x_1|\leq |x_2|\;\textbf{then}< x_2, x_1 >\;\textbf{else}\;< x_1, x_2 >))$$
</section>
<section>
<h3>The Optimizer</h3>
Starting with $e$...
<ol>
<li>For every possible rewrite of $e$:</li>
<li style="padding-left: 40px">Use a linear optimizer to find the best $k$s</li>
<li style="padding-left: 40px">If the rewrite improved the cost then recur.</li>
</ol>
</section>
<section>
<h3>Legorithmics</h3>
<ul>
<li>Define your algorithms once.</li>
<li>Define your hardware spec once.</li>
<li>Let an automatic tool fit the algoroithm to the hardware!</li>
</ul>
</section>
</section>

7
src/teaching/cse-662/2019fa/slide/2019-10-10-Legorithmics.erb → src/teaching/cse-662/2019fa/slide/2019-10-22-Legorithmics.erb
View file

@ -1,15 +1,10 @@
---
template: templates/cse662_2019_slides.erb
title: Legorithmics
date: October 10
date: October 22
---
<section>
<section>
<h2>Legorithmics</h2>
<h4>October 23, 2017</h4>
</section>
<section>
<svg width=700 height=450>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper6.png"