ODIn/Website
9
0
Fork 1
Code Pull requests Releases Wiki Activity

Papers

Browse source
This commit is contained in:
Oliver Kennedy 2019-10-10 19:05:35 -04:00
parent dd18ccd93c
commit df9f064fac
Signed by: okennedy
GPG key ID: 3E5F9B3ABD3FDB60
7 changed files with 475 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

13
src/teaching/cse-662/2019fa/index.md
View file

@ -36,7 +36,7 @@ about the group's report, deliverables, and any related tools and technology.
their final project. A roughly 5-page report should outline the group’s their final project. A roughly 5-page report should outline the group’s
proposed design, any algorithms or data structures being introduced, as well proposed design, any algorithms or data structures being introduced, as well
as a strategy for evaluating the resulting project against the current state as a strategy for evaluating the resulting project against the current state
of the art. This report and presentation constitute 35% of the final grade. of the art. This report and presentation constitute 15% of the final grade.
3. At the final stage, students are expected to provide a roughly 5-page report 3. At the final stage, students are expected to provide a roughly 5-page report
detailing their project, any algorithms or data structures developed, and detailing their project, any algorithms or data structures developed, and
evaluating their project against any comparable state of the art systems and evaluating their project against any comparable state of the art systems and
@ -53,6 +53,10 @@ about the group's report, deliverables, and any related tools and technology.
* Oliver Kennedy (Capen 212; Office Hours TBD) * Oliver Kennedy (Capen 212; Office Hours TBD)
* **Time**: TR 5:00-6:20 PM * **Time**: TR 5:00-6:20 PM
* **Location**: NSC 201 * **Location**: NSC 201
* **Checkpoint Submissions**
* **Checkpoint 1**: Sept. 25
* **Checkpoint 2**: Oct. 23
* **Checkpoint 3**: Dec. 6
--- ---
@ -92,6 +96,13 @@ After the taking the course, students should be able to:
* **Oct 1** - Interpretable Deep Learning ([reading 1](https://ieeexplore-ieee-org.gate.lib.buffalo.edu/abstract/document/8022871) | [reading 2](https://dl-acm-org.gate.lib.buffalo.edu/citation.cfm?Id=2939778)) * **Oct 1** - Interpretable Deep Learning ([reading 1](https://ieeexplore-ieee-org.gate.lib.buffalo.edu/abstract/document/8022871) | [reading 2](https://dl-acm-org.gate.lib.buffalo.edu/citation.cfm?Id=2939778))
* **Oct 3** - SkyServer on MonetDB presented by SQLatin ([reading](https://ieeexplore-ieee-org.gate.lib.buffalo.edu/abstract/document/4274958/) | slides) * **Oct 3** - SkyServer on MonetDB presented by SQLatin ([reading](https://ieeexplore-ieee-org.gate.lib.buffalo.edu/abstract/document/4274958/) | slides)
* **Oct 8** - Software Transactional Memory ([reading](https://dl.acm.org/citation.cfm?id=1378582)) * **Oct 8** - Software Transactional Memory ([reading](https://dl.acm.org/citation.cfm?id=1378582))
* **Oct 10** - Skyserver (continued)
* **Oct 15** - NoDB / RAW ([paper 1](http://dl-acm-org.gate.lib.buffalo.edu/citation.cfm?id=2213864) | [paper 2](http://www.vldb.org/pvldb/vol7/p1119-karpathiotakis.pdf))
* **Oct 17** - Group Presentations
* **Oct 22** - Legorithmics ([paper](http://infoscience.epfl.ch/record/186017/files/main-final.pdf))
* **Oct 24** - Streaming ([paper](http://www.cs.cornell.edu/johannes/papers/2007/2007-CIDR-Cayuga.pdf))
* **Oct 29** - Scan Sharing ([paper](http://dl-acm-org.gate.lib.buffalo.edu/citation.cfm?id=1687707))
* **Oct 31** - Group Presentations
--- ---

BIN
src/teaching/cse-662/2019fa/slide/2019-10-08-STM.pptx Normal file
View file

Binary file not shown.

463
src/teaching/cse-662/2019fa/slide/2019-10-15-Legorithmics.erb Normal file
View file

@ -0,0 +1,463 @@
---
template: templates/cse662_2019_slides.erb
title: Legorithmics
date: October 15
---
<section>
<section>
<h2>Legorithmics</h2>
<h4>October 23, 2017</h4>
</section>
<section>
<svg width=700 height=450>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper6.png"
x="50" y="0"
width="446" height="183"
border="1"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper3.png"
x="250" y="60"
width="424" height="183"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper5.png"
x="0" y="160"
width="422" height="190"
/>
<image class="fragment" xlink:href="graphics/Legorithmics-Paper4.png"
x="200" y="220"
width="473" height="229"
/>
</svg>
<p class="fragment"><b>Same great algorithms, <br/> awesome new hardware flavor</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>Hardware adaptation uses standard transformations:
<ul>
<li class="fragment"><b>Batching</b>: Prefetch blocks of data to avoid random seeks.</li>
<li class="fragment"><b>Partitioning</b>: Group related blocks of data to minimize cross-partition work.</li>
<li class="fragment"><b>Reordering</b>: Cluster data accesses for better cache locality.</li>
</ul>
</p>
<p class="fragment">The hard part is picking <b>where to apply the transformations</b> and selecting <b>values for the transformation's parameters</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>Key Insights:
<ul>
<li>The basic algorithms haven't changed since the 70s.</li>
<li>Hardware changes very slowly</li>
<li class="fragment">You have as much time as you need to design a hardware-specific algorithm</li>
</ul>
</p>
<p class="fragment"><b>Automate the search!</b></p>
</section>
<section>
<h3>Adapting Software to Hardware</h3>
<p>What do we need?</p>
<ul>
<li class="fragment">A way to describe the algorithm.</li>
<li class="fragment">A way to describe the hardware.</li>
<li class="fragment">A cost model.</li>
<li class="fragment">Transformation rules.</li>
<li class="fragment">A (possibly exp-time) optimizer</li>
</ul>
</section>
</section>
<section>
<section>
<h3>Typesystems</h3>
<p>A collection of rules that assign a property called a <i>type</i> to the parts of a computer program: variables, expressions, etc...<br/><attribution>[Wikipedia]</attribution></p>
</section>
<section>
<h3>Typesystems</h3>
<p>A typesystem allows you to:
<ul>
<li class="fragment">Define interfaces between different parts of a program.</li>
<li class="fragment">Check that these parts have been connected consistently.</li>
<li class="fragment"><span class="fragment highlight-blue">Define global properties in terms of local properties.</span></li>
</ul>
</p>
</section>
<section>
<h3>Typesystems</h3>
<br/>
<p><b>A type</b><br/>
($\tau := D\;|\;[\tau]\;|<\tau,\tau>\;|\;\tau\rightarrow\tau$)
</p>
<br/>
<p><b>A set of inference rules</b><br/>
($\frac{e\;:\;\tau}{[e]\;:\;[\tau]}$)
</p>
<br/>
<p class="fragment">These example types are part of the monad algebra</p>
</section>
<section>
<h3>Types</h3>
<table>
<tr><th>Type</th><th>Meaning</th></tr>
<tr><td>$D$</td><td>Primitive Type (int, float, etc...)</td></tr>
<tr><td>$[\tau]$</td><td>An array of elements with type $\tau$</td></tr>
<tr><td>$<\tau_1,\tau_2>$</td><td>A pair of elements of types $\tau_1$ and $\tau_2$.</td></tr>
<tr><td>$\tau_1\rightarrow\tau_2$</td><td>A function with one argument of type $\tau_1$ and one return value of type $\tau_2$.</td></tr>
</table>
</section>
<section>
<h3>Type Examples</h3>
<br/><div class="fragment">$[ < int,float > ]$</div>
<br/><div class="fragment">$[int] \rightarrow float$</div>
<br/><div class="fragment">$ < [int], [int] >\; \rightarrow [ < int,int > ]$</div>
</section>
<section>
<h3>Inference Rules</h3>
<br/>
<p>Defined over a language of expressions like $f(a, b)$.</p>
$$\frac{a : \tau_a\;\;\;b : \tau_b}{f(a,b):\tau_f(\tau_a, \tau_b)}$$
<p class="fragment">If expression $a$ has type $\tau_a$ and expression $b$ has type $\tau_b$...</p>
<p class="fragment">...then expression $f(a, b)$ has type $\tau_f(\tau_a, \tau_b)$.</p>
</section>
<section>
<h3>Inference Examples</h3>
<br/><div class="fragment">
$\frac{e: \tau}{[e] : [\tau]}$
</div>
<br/><div class="fragment">
$\frac{c: Bool\;\;\;e_1:\tau\;\;\;e_2:\tau}{(\textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2)\;:\; \tau}$
</div>
<br/><div class="fragment">
$\frac{e: < \tau_1, \tau_2 >\;\;\;i \in \{1,2\}}{e.i\; :\; \tau_i}$
</div>
</section>
</section>
<section>
<section>
<h3>Monad Algebra</h3>
<p>A primitive language for describing data processing.</p>
<table>
<tr><th>Operator</th><th>Meaning</th></tr>
<tr><td>$\lambda x.e$</td><td>Define a function with body $e$ that uses variable $x$.</td></tr>
<tr><td>$e_1\;e_2$</td><td>Apply the function defined by $e_1$ to the value obtained from $e_2$.</td></tr>
<tr><td>$ \textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2$</td><td>If $c$ is true then evaluate $e_1$, and otherwise evaluate $e_2$.</td></tr>
</table>
</section>
<section>
<h3>Monad Algebra</h3>
<table>
<tr><th>Operator</th><th>Meaning</th></tr>
<tr><td>$ < e_1, e_2 >$</td><td>Construct a tuple from $e_1$ and $e_2$.</td></tr>
<tr><td>$ e.i $</td><td>Extract attribute $i$ from the tuple $e$.</td></tr>
<tr><td>$ [e] $</td><td>Construct a single-element array from e.</td></tr>
<tr><td>$ [] $</td><td>Construct an empty array.</td></tr>
<tr><td>$ e_1 \sqcup e_2 $</td><td>Concatenate the arrays $e_1$ and $e_2$.</td></tr>
</table>
</section>
<section>
$$\textbf{flatMap}(f : \tau_1 \rightarrow [\tau_2])(e : [\tau_1])$$
<p class="fragment">Apply function $f$ to every element of array $e$. Concatenate all of the arrays returned by $f$.</p>
$$\textbf{foldL}(c : \tau_2, f : < \tau_2, \tau_1 >)(e : [\tau_1])$$
<p class="fragment">Apply function $f$ to every element of array $e$, with each invocation passing its return value to the next call (e.g., aggregation)<p>
$$\textbf{for}(xB : [\tau_1] [k] \leftarrow e_{in} : [\tau_1])(e_{loop} : [\tau_2])$$
<p class="fragment">Extract blocks of size $k$ from $e_{in}$. For each block compute a <b>flatMap</b> using expression $e_{loop}$.<p>
</section>
<section>
<h3>Example - Average</h3>
<div style="text-align: left">
<span class="fragment" data-fragment-index=6>$(\lambda tot.(tot.1 / tot.2))$</span><br/>
&nbsp;&nbsp;&nbsp;&nbsp;
<span class="fragment" data-fragment-index=1>$(\textbf{foldL}($
<span class="fragment" data-fragment-index=5>$< 0, 0 >,$</span>
<span class="fragment" data-fragment-index=2>$(\lambda < a, x >.<$
<span class="fragment" data-fragment-index=3>$a.1 + x$</span>
$,$
<span class="fragment" data-fragment-index=4>$a.2 + 1$</span>
$ > $</span>
$))$</span>
</div>
<p class="fragment" data-fragment-index=1>Fold implements aggregation</p>
<p class="fragment" data-fragment-index=2>Fold takes a 'previous' $a$ and a 'current' $x$</p>
<p class="fragment" data-fragment-index=3>We need a sum<span class="fragment" data-fragment-index=4>, and a count</span></p>
<p class="fragment" data-fragment-index=5>Initial sum and count are both 0</p>
<p class="fragment" data-fragment-index=6>Postprocess with division ($\lambda$ creates a variable $tot$)</p>
</section>
</section>
<section>
<section>
<h3>Cost Estimation</h3>
<p>We need...
<ul>
<li class="fragment">Cardinality estimation</li>
<li class="fragment">A model of IO</li>
<li class="fragment">IO Cost relative to cardinality</li>
</ul></p>
</section>
<section>
<h3>Cardinality Estimation</h3>
<p><b>Basic Approach:</b> Define a second <i>type</i> for tracking data sizes</p>
$$\alpha\;:=\;[\alpha]^x\;|\;< \alpha_1, \alpha_2 >|\;c$$
<br/>
<div class="fragment">
<p>e.g., $[ < 1, [1]^y > ]^x$ corresponds to:
<ul>
<li>an array with $x$ elements, where each element consists of...</li>
<li> a value of fixed-size 1, and...</li>
<li> an array of $y$ fixed-size values.</li>
</ul></p>
</div>
</section>
<section>
<h3>Cardinality Estimation</h3>
<ul>
<li>$size(c) = c$</li>
<li>$size([\alpha]^x) = x \cdot size(\alpha)$</li>
<li>$size( < \alpha_1, \alpha_2 >) = size(\alpha_1) + size(\alpha_2)$</li>
</ul>
</section>
<section>
<h3>Cardinality Estimation</h3>
<p>$R(\Gamma, e)$ computes the cardinality type for $e$</p>
<p>$\Gamma : x \rightarrow \alpha$ is a context / scope</p>
</section>
<section>
<h3>For Loops</h3>
<br/>
$$R\left(\Gamma, \textbf{for}(x [k] \leftarrow e_1)\; e_2\right) := $$
<p>
<span class="fragment" data-fragment-index=2>$\frac{cardinality(R(\Gamma, e_1))}{k}\cdot$</span>
<span class="fragment" data-fragment-index=1>$R(\Gamma', e_2)$</span>
</p>
<br/>
<div class="fragment" data-fragment-index=3>$\Gamma' := \Gamma \cup \{x \mapsto [sizeofElement(R(\Gamma, e_1))]^k\}$</div>
<br/>
<p class="fragment" data-fragment-index=1>The cardinality is based on that of $e_2$</p>
<p class="fragment" data-fragment-index=2>Repeated once for every time through the loop</p>
<p class="fragment" data-fragment-index=3>And $e_2$ is evaluated in the context of a $k$ element array.</p>
</section>
<section>
<h3>If Then Else</h3>
<br/>
$$R\left(\Gamma, \textbf{if}\;c\;\textbf{then}\;e_1\;\textbf{else}\;e_2\right) := $$
<p>
$max(R\left(\Gamma, e_1\right), R\left(\Gamma, e_2\right))$
</p>
<p class="fragment">Pessimistic assumption of biggest possible size. <br/> Avoids needing to estimate $p(c = true)$.</p>
</section>
</section>
<section>
<section>
<h3>Example: Block-Nested-Loop Join</h3>
<div style="text-align: left;">
<div style="float: right"># Loop over blocks in outer rel.</div>
<div style="margin-left: 0px">$\textbf{for}( xB [k_1] \leftarrow R )$</div>
<div style="float: right"># Loop over blocks in inner rel.</div>
<div style="margin-left: 30px">$\textbf{for}( yB [k_2] \leftarrow S )$</div>
<div style="float: right"># Loop over elems in outer block.</div>
<div style="margin-left: 60px">$\textbf{for}( x \leftarrow xB )$</div>
<div style="float: right"># Loop over elems in inner block.</div>
<div style="margin-left: 90px">$\textbf{for}( y \leftarrow yB )$</div>
<div style="float: right"># Join test.</div>
<div style="margin-left:120px">$\textbf{if}\;joinCond(x, y)$</div>
<div style="float: right"># Add pair if success.</div>
<div style="margin-left:150px">$\textbf{then}\;[< x, y >]$</div>
<div style="float: right"># Add nothing if not.</div>
<div style="margin-left:150px">$\textbf{else}\;[]$</div>
</div>
</section>
<section>
<table>
<tr><th>Expression</th><th>Context</th><th>Result Size</th></tr>
<tr><td>$\textbf{for}( xB [k_1] \leftarrow R )$</td>
<td class="fragment"
data-fragment-index=1
style="text-align: left">$\Gamma_1 = R \mapsto [1]^x, S \mapsto [1]^y$</td>
<td class="fragment"
data-fragment-index=14
>$[ < 1, 1 > ]^{\frac{x}{k_1} \cdot \frac{y}{k_2} \cdot k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( yB [k_2] \leftarrow S )$</td>
<td class="fragment"
data-fragment-index=2
style="text-align: left">$\Gamma_2 = \Gamma_1 \cup xB \mapsto [1]^{k_1}$</td>
<td class="fragment"
data-fragment-index=13
>$[ < 1, 1 > ]^{\frac{y}{k_2} \cdot k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( x \leftarrow xB )$</td>
<td class="fragment"
data-fragment-index=3
style="text-align: left">$\Gamma_3 = \Gamma_2 \cup yB \mapsto [1]^{k_2}$</td>
<td class="fragment"
data-fragment-index=12
>$[ < 1, 1 > ]^{k_1 \cdot k_2}$</td></tr>
<tr><td>$\textbf{for}( y \leftarrow yB )$</td>
<td class="fragment"
data-fragment-index=4
style="text-align: left">$\Gamma_4 = \Gamma_3 \cup x \mapsto 1$</td>
<td class="fragment"
data-fragment-index=11
>$[ < 1, 1 > ]^{k_2}$</td></tr>
<tr><td>$\textbf{if}\;joinCond(x, y)$</td>
<td class="fragment"
data-fragment-index=5
style="text-align: left">$\Gamma_5 = \Gamma_4 \cup y \mapsto 1$</td>
<td class="fragment"
data-fragment-index=10
>$[ < 1, 1 > ]^1$</td></tr>
<tr><td>$\textbf{then}\;[< x, y >]$</td>
<td class="fragment"
data-fragment-index=6
style="text-align: left">$\Gamma_5$</td>
<td class="fragment"
data-fragment-index=9
>$[ < 1, 1 > ]^1$</td></tr>
<tr><td>$\textbf{else}\;[]$</td>
<td class="fragment"
data-fragment-index=6
style="text-align: left">$\Gamma_5$</td>
<td class="fragment"
data-fragment-index=8
>$0$</td></tr>
</table>
</section>
</section>
<section>
<section>
<h3>IO Model</h3>
<p>IO Costs have 2 components:
<small><ul>
<li>$InitCom$: The cost of initializing a connection (e.g., seek time).</li>
<li>$UnitTr$: The cost of transferring one unit of data.</li>
</ul></small></p>
<p>Costs are defined for every pair of memory hierarchy levels:
<small><ul>
<li>$UnitTr(HDD \rightarrow RAM)$ is the cost of reading from HDD into Ram.</li>
<li>$InitCom(RAM \rightarrow HDD)$ is the cost of seeking to a write from Ram onto a HDD.</li>
</small></ul></p>
</section>
<section>
<table>
<tr><th>Expression</th><th>Result Size</th><th>HDD&nbsp;to&nbsp;RAM</th><th>RAM&nbsp;to&nbsp;HDD</th></tr>
<tr><td><small>$\textbf{for}( xB [k_1] \leftarrow R )$</small></td>
<td><small>$[ < 1, 1 > ]^{\frac{x}{k_1} \cdot \frac{y}{k_2} \cdot k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=7>$x+\frac{x}{k_1}y$</td>
<td class="fragment" data-fragment-index=4>$2xy$</td>
</tr>
<tr><td><small>$\textbf{for}( yB [k_2] \leftarrow S )$</small></td>
<td><small>$[ < 1, 1 > ]^{\frac{y}{k_2} \cdot k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=6>$y$</td>
<td class="fragment" data-fragment-index=4>$2k_1y$</td>
</tr>
<tr><td><small>$\textbf{for}( x \leftarrow xB )$</small></td>
<td><small>$[ < 1, 1 > ]^{k_1 \cdot k_2}$</small></td>
<td class="fragment" data-fragment-index=5>$0$</td>
<td class="fragment" data-fragment-index=4>$2k_1k_2$</td>
</tr>
<tr><td><small>$\textbf{for}( y \leftarrow yB )$</small></td>
<td><small>$[ < 1, 1 > ]^{k_2}$</small></td>
<td class="fragment" data-fragment-index=5>$0$</td>
<td class="fragment" data-fragment-index=3>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{if}\;joinCond(x, y)$</small></td>
<td><small>$[ < 1, 1 > ]^1$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=3>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{then}\;[< x, y >]$</small></td>
<td><small>$[ < 1, 1 > ]^1$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=2>$(1+1)k_2$</td>
</tr>
<tr><td><small>$\textbf{else}\;[]$</small></td>
<td><small>$0$</small></td>
<td class="fragment" data-fragment-index=1>$0$</td>
<td class="fragment" data-fragment-index=1>$0$</td>
</tr>
</table>
<p>HDD: $R, S, Result$<span style="margin-left: 100px">&nbsp;</span>RAM: $x, xB, y, yB$</p>
</section>
</section>
<section>
<section>
<h3>Rewrite Rules</h3>
<h4>Batching</h4>
$$for(x [1] \leftarrow R)\; e \Rightarrow for(xB [k] \leftarrow R)\; for(x [1] \leftarrow xB)\; e$$
<h4 style="margin-top: 30px">Reordering Iterators</h4>
$$for(x_1 [k_1] \leftarrow R_1)\;for(x_2 [k_2] \leftarrow R_2) \Rightarrow$$
$$for(x_2 [k_2] \leftarrow R_2)\;for(x_1 [k_1] \leftarrow R_1)$$
<h4 style="margin-top: 30px">Size-Dependent, Commutative Functions</h4>
$$f \Rightarrow (\lambda< x_1, x_2>.f(\textbf{if}\;|x_1|\leq |x_2|\;\textbf{then}< x_1, x_2 >\;\textbf{else}\;< x_2, x_1 >))$$
$$f \Rightarrow (\lambda< x_1, x_2>.f(\textbf{if}\;|x_1|\leq |x_2|\;\textbf{then}< x_2, x_1 >\;\textbf{else}\;< x_1, x_2 >))$$
</section>
<section>
<h3>The Optimizer</h3>
Starting with $e$...
<ol>
<li>For every possible rewrite of $e$:</li>
<li style="padding-left: 40px">Use a linear optimizer to find the best $k$s</li>
<li style="padding-left: 40px">If the rewrite improved the cost then recur.</li>
</ol>
</section>
<section>
<h3>Legorithmics</h3>
<ul>
<li>Define your algorithms once.</li>
<li>Define your hardware spec once.</li>
<li>Let an automatic tool fit the algoroithm to the hardware!</li>
</ul>
</section>
</section>

BIN
src/teaching/cse-662/2019fa/slide/graphics/Legorithmics-Paper3.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 264 KiB

BIN
src/teaching/cse-662/2019fa/slide/graphics/Legorithmics-Paper4.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 397 KiB

BIN
src/teaching/cse-662/2019fa/slide/graphics/Legorithmics-Paper5.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 215 KiB

BIN
src/teaching/cse-662/2019fa/slide/graphics/Legorithmics-Paper6.png Normal file
View file

Binary file not shown.
Side by side

After

Width:  |  Height:  |  Size: 244 KiB