pull/2/head
Oliver Kennedy 2022-09-29 22:49:26 -04:00
parent 1b4642d9e3
commit a122de8bd8
Signed by: okennedy
GPG Key ID: 3E5F9B3ABD3FDB60
2 changed files with 435 additions and 137 deletions

View File

@ -283,140 +283,3 @@ textbook: Ch. 15
<p class="fragment">$= 3.5 + 3.5 = 7$</p>
</section>
<section>
$$T(n) = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
T(0) + T(n-1) + \Theta(n) & \textbf{if } n > 1 \wedge X = 1\\
T(1) + T(n-2) + \Theta(n) & \textbf{if } n > 1 \wedge X = 2\\
T(2) + T(n-3) + \Theta(n) & \textbf{if } n > 1 \wedge X = 3\\
..\\
T(n-2) + T(1) + \Theta(n) & \textbf{if } n > 1 \wedge X = n-1\\
T(n-1) + T(0) + \Theta(n) & \textbf{if } n > 1 \wedge X = n\\
\end{cases}$$
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
E[T(X-1) + T(n-X)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
</section>
<section>
$$\begin{cases}
T(0) + T(n-1) + \Theta(n) & \textbf{if } X = 1\\
T(1) + T(n-2) + \Theta(n) & \textbf{if } X = 2\\
T(2) + T(n-3) + \Theta(n) & \textbf{if } X = 3\\
..\\
T(n-2) + T(1) + \Theta(n) & \textbf{if } X = n-1\\
T(n-1) + T(0) + \Theta(n) & \textbf{if } X = n\\
\end{cases}$$
<p>There's a symmetry: The left varies from $T(0)$ to $T(n-1)$, and visa versa on the right.</p>
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
E[2 \cdot T(X-1)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
<p>&nbsp;</p>
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
2 E[T(X-1)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
<p>&nbsp;</p>
</section>
<section>
<p>&nbsp;</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
2 E\left[\sum_{i=0}^{n-1} \frac{1}{n} T(i) \right] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
<p class="fragment">Each $T(X-1)$ is <i>independent</i>.</p>
</section>
<section>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise}
\end{cases}$$
</section>
<section>
<p>Back to induction...</p>
<p><b>Hypothesis: </b> $E[T(n)] \in O(n \log(n))$</p>
</section>
<section>
<p><b>Base Case: </b> $E[T(1)] \lec c(1 \log(1))$</p>
<p class="fragment">$$E[T(1)] \leq c \cdot (1 \log(1))$$</p>
<p class="fragment">$$E[T(1)] \leq c \cdot (1 \cdot 0)$$</p>
<p class="fragment">$$E[T(1)] \not \leq 0$$</p>
</section>
<section>
<p><b>Base Case (take Two): </b> $E[T(2)] \leq c(2 \log(2))$</p>
<p class="fragment">$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$</p>
<p class="fragment">$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$</p>
<p class="fragment">$$T(0) + T(1) + 2c_1 \leq 2c$$</p>
<p class="fragment">$$2c_0 + 2c_1 \leq 2c$$</p>
<p class="fragment">True for any $c \geq c_0 + c_1$</p>
</section>
<section>
<p><b>Assume: </b> $E[T(n')] \leq c(n' \log(n'))$ for <b>all</b> $n' < n$</p>
<p><b>Show: </b> $E[T(n)] \leq c(n \log(n))$</p>
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
</section>
<section>
<p>$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c_1 \leq c\log(n)$$</p>
</section>
<section>
<p>$E[T_{quicksort}(n)] = O(n\log(n))$</p>
<p>So is Quicksort $O(n\log(n))$? <span class="fragment"><b>No!</b></span></p>
</section>
<section>
<h4 class="slide_title">What <b>guarantees</b> do you get?</h4>
<dl>
<dt>$f(n)$ is a Tight Bound</dt>
<dd>The algorithm <b>always</b> runs in $cf(n)$ steps</dd>
<dt>$f(n)$ is a Worst-Case Bound</dt>
<dd>The algorithm <b>always</b> runs at-most $cf(n)$ steps</dd>
<dt>$f(n)$ is an Amortized Worst-Case Bound</dt>
<dd>$n$ invocations of the algorithm <b>always</b> run in $cnf(n)$ steps</dd>
<dt>$f(n)$ is an Average Bound</dt>
<dd>🤷</dd>
</dl>
</section>
<section>
<h4 class="slide_title">Next time...</h4>
<p>Special <tt>Seq</tt>uences: <tt>Stack</tt>, <tt>Queue</tt></p>
</section>

View File

@ -0,0 +1,435 @@
---
template: templates/cse250_2022_slides.erb
title: Average Runtime, Stacks, Queues
date: Sept 30, 2022
textbook: Ch. 15
---
<section>
<h4 class="slide_title">QuickSort</h4>
<ol>
<li>Pick a value at random as a $pivot$.</li>
<li>Swap values until the array is subdivided into...<ul>
<li>$low$: array elements that are $\leq pivot$</li>
<li>$pivot$</li>
<li>$high$: array elements that are $> pivot$</li>
</ul></li>
<li>Recursively sort $low$</li>
<li>Recursively sort $high$</li>
</ol>
</section>
<section>
<p>What's the worst-case runtime?</p>
<p class="fragment">$$T_{quicksort}(n) \in O(n^2)$$</p>
<p class="fragment">... but this isn't really representative of typical behavior</p>
</section>
<section>
<p>If $X$ represents a random (numerical) outcome, the average over all possibilities is the <i>expectation</i> of $X$, or $E[X]$</p>
<p>
$$E[X] = \sum_{i} P_i \cdot X_i$$
</p>
<ol>
<li>$E[X + Y] = E[X] + E[Y]$ (always)</li>
<li>$E[X \cdot Y] = E[X] \cdot E[Y]$ (If $X$ and $Y$ are <i>independent</i>)</li>
</ol>
</section>
<section>
$$T(n) = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
T(0) + T(n-1) + \Theta(n) & \textbf{if } n > 1 \wedge X = 1\\
T(1) + T(n-2) + \Theta(n) & \textbf{if } n > 1 \wedge X = 2\\
T(2) + T(n-3) + \Theta(n) & \textbf{if } n > 1 \wedge X = 3\\
..\\
T(n-2) + T(1) + \Theta(n) & \textbf{if } n > 1 \wedge X = n-1\\
T(n-1) + T(0) + \Theta(n) & \textbf{if } n > 1 \wedge X = n\\
\end{cases}$$
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
E[T(X-1) + T(n-X)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
E[T(X-1)] + E[T(n-X)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
</section>
<section>
<p>$$E[T(X-1)]$$</p>
<p class="fragment">$= \sum_{i=1}^{n} P_i \cdot T(X_i-1)$</p>
<p class="fragment">$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(i-1)$ <span style="font-size: 80%; font-weight: bold; margin-left: 50px;">($T(0)$ up to $T(n-1)$)</span></p>
<p class="fragment">$= \sum_{i=1}^{n} \frac{1}{n} \cdot T(n-i)$ <span style="font-size: 80%; font-weight: bold; margin-left: 50px;">($T(n-1)$ down to $T(0)$)</span></p>
<p class="fragment">$= E[T(n-X)]$</p>
</section>
<section>
<p>We pick the $X$th largest element as a pivot</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
2 E[T(X-1)] + \Theta(n) & \textbf{otherwise}
\end{cases}$$
<p class="fragment">Each $T(X-1)$ is <i>independent</i>.</p>
</section>
<section>
<p>&nbsp;</p>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
2 \left(\sum_{i=1}^{n} \frac{1}{n} E[T(i-1)] \right) + \Theta(n) & \textbf{otherwise}
\end{cases}$$
<p>&nbsp;</p>
</section>
<section>
$$E[T(n)] = \begin{cases}
\Theta(1) & \textbf{if } n \leq 1\\
\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise}
\end{cases}$$
</section>
<section>
<p>Back to induction...</p>
<p><b>Hypothesis: </b> $E[T(n)] \in O(n \log(n))$</p>
</section>
<section>
<p><b>Base Case: </b> $E[T(1)] \leq c(1 \log(1))$</p>
<p class="fragment">$$E[T(1)] \leq c \cdot (1 \cdot 0)$$</p>
<p class="fragment">$$E[T(1)] \not \leq 0$$</p>
</section>
<section>
<p><b>Base Case (take Two): </b> $E[T(2)] \leq c(2 \log(2))$</p>
<p class="fragment">$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$</p>
<p class="fragment">$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$</p>
<p class="fragment">$$T(0) + T(1) + 2c_1 \leq 2c$$</p>
<p class="fragment">$$2c_0 + 2c_1 \leq 2c$$</p>
<p class="fragment">True for any $c \geq c_0 + c_1$</p>
</section>
<section>
<p><b>Assume: </b> $E[T(n')] \leq c(n' \log(n'))$ for <b>all</b> $n' < n$</p>
<p><b>Show: </b> $E[T(n)] \leq c(n \log(n))$</p>
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
</section>
<section>
<div style="font-size: 90%">
<p>$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$</p>
<p class="fragment">$$c_1 \leq c\log(n)$$</p>
</div>
</section>
<section>
<p>$E[T_{quicksort}(n)] = O(n\log(n))$</p>
<p>So is Quicksort $O(n\log(n))$? <span class="fragment"><b>No!</b></span></p>
</section>
<section>
<h4 class="slide_title">What <b>guarantees</b> do you get?</h4>
<dl style="font-size: 90%">
<dt>$f(n)$ is a Tight Bound</dt>
<dd>The algorithm <b>always</b> runs in <b>exactly</b> $cf(n)$ steps</dd>
<dt>$f(n)$ is a Worst-Case Bound</dt>
<dd>The algorithm <b>always</b> runs <b>at-most</b> $cf(n)$ steps</dd>
<dt>$f(n)$ is an Amortized Worst-Case Bound</dt>
<dd><b>$n$ invocations</b> of the algorithm <b>always</b> run in $cnf(n)$ steps</dd>
<dt>$f(n)$ is an Average Bound</dt>
<dd>🤷</dd>
</dl>
</section>
<section>
<h4 class="slide_title">mutable.Seq</h4>
<dl>
<dt><tt>mutable.IndexedSeq</tt> (e.g., Array)</dt>
<dd>Efficient apply(), update()</dd>
<dt><tt>mutable.Buffer</tt> (e.g., ArrayBuffer, ListBuffer)</dt>
<dd>Efficient apply(), update(), append()</dd>
<div class="fragment">
<dt><tt>mutable.Stack</tt></dt>
<dd>Efficient push(), pop(), top()</dd>
<dt><tt>mutable.Queue</tt></dt>
<dd>Efficient enqueue(), dequeue(), head()</dd>
</div>
</dl>
</section>
<section>
<h4 class="slide_title">Stacks</h4>
<p>A stack of objects on top of one another.</p>
<dl>
<dt>Push</dt>
<dd>Put a new object on top of the stack</dd>
<dt>Pop</dt>
<dd>Remove the object on top of the stack</dd>
<dt>Top</dt>
<dd>Peek at what's on top of the stack</dd>
</dl>
</section>
<section>
<style type="text/css">
#demo_stack {
height: 400px;
width: 300px;
border: solid 1px black;
margin-left: auto;
margin-right: auto;
overflow-y: hidden;
}
#demo_stack .stack_element {
width: 200px;
color: lightgray;
background: black;
margin-left: auto;
margin-right: auto;
margin-top: 5px;
}
</style>
<button onclick="push()">Push</button>
<button onclick="pop()">Pop</button>
<div id="demo_stack"></div>
<script type="text/javascript">
var stack_demo = document.getElementById("demo_stack")
var stack_counter = 0
function push(){
var elem = document.createElement('div')
elem.classList.add("stack_element")
elem.innerHTML = ""+(stack_counter++)
stack_demo.prepend(elem)
}
function pop(){
var first = stack_demo.firstElementChild
if(first){ first.remove() }
}
</script>
</section>
<section>
<h4 class="slide_title">Stacks in Practice</h4>
<ul>
<li>Storing function variables in a "call stack"</li>
<li>Certain types of parsers ("context free")</li>
<li>Backtracking search</li>
<li>Reversing sequences</li>
</ul>
</section>
<section>
<h4 class="slide_title">Stacks</h4>
<pre><code class="scala">
trait Stack[A] {
def push(element: A): Unit
def top: A
def pop: A
}
</code></pre>
</section>
<section>
<h4 class="slide_title">ListStack</h4>
<pre><code class="scala">
class ListStack[A] extends Stack[A] {
val _store = new SinglyLinkedList()
def push(element: A): Unit =
_store.prepend(element)
def top: A =
_store.head
def pop: A =
_store.remove(0)
}
</code></pre>
<p class="fragment">What's the runtime?</p>
</section>
<section>
<h4 class="slide_title">ArrayBufferStack</h4>
<pre><code class="scala">
class ArrayBufferStack[A] extends Stack[A] {
val _store = new ArrayBuffer()
def push(element: A): Unit =
_store.append(element)
def top: A =
_store.last
def pop: A =
_store.remove(store.length-1)
}
</code></pre>
<p class="fragment">What's the runtime?</p>
</section>
<section>
<h4 class="slide_title">Stack</h4>
<p>Scala's <tt>Stack</tt> implementation is based on <tt>ArrayBuffer</tt> (<tt>ArrayDequeue</tt>); Keeping memory together is worth the overhead of <u>amortized</u> $O(1)$.</p>
</section>
<section>
<h4 class="slide_title">Queue</h4>
<p>Outside of the US, "queueing" is lining up.</p>
<dl>
<dt>Enqueue</dt>
<dd>Put a new object at the end of the queue</dd>
<dt>Dequeue</dt>
<dd>Remove the next object in the queue</dd>
<dt>Head</dt>
<dd>Peek at the next object in the queue</dd>
</dl>
</section>
<section>
<style type="text/css">
#demo_queue {
height: 400px;
width: 300px;
border: solid 1px black;
margin-left: auto;
margin-right: auto;
overflow-y: hidden;
}
#demo_queue .queue_element {
width: 200px;
color: lightgray;
background: black;
margin-left: auto;
margin-right: auto;
margin-top: 5px;
}
</style>
<button onclick="enqueue()">Enqueue</button>
<button onclick="dequeue()">Dequeue</button>
<div id="demo_queue"></div>
<script type="text/javascript">
var queue_demo = document.getElementById("demo_queue")
var queue_counter = 0
function enqueue(){
var elem = document.createElement('div')
elem.classList.add("queue_element")
elem.innerHTML = ""+(queue_counter++)
queue_demo.append(elem)
}
function dequeue(){
var first = queue_demo.firstElementChild
if(first){ first.remove() }
}
</script>
</section>
<section>
<h4 class="slide_title">Queues vs Stacks</h4>
<dl>
<dt>Queue</dt>
<dd>First in, First out (FIFO)</dd>
<dt>Stack</dt>
<dd>Last in, First Out (LIFO / FILO)</dd>
</dl>
</section>
<section>
<h4 class="slide_title">Queues in Practice</h4>
<ul>
<li>Delivering network packets, emails, twokstagrams</li>
<li>Scheduling CPU Cycles</li>
<li>Deferring long-running tasks</li>
</ul>
</section>
<section>
<h4 class="slide_title">Queues</h4>
<pre><code class="scala">
trait Queue[A] {
def enqueue(element: A): Unit
def dequeue: A
def head: A
}
</code></pre>
</section>
<section>
<h4 class="slide_title">ListQueue</h4>
<pre><code class="scala">
class ListQueue[A] extends Queue[A] {
val _store = new DoublyLinkedList()
def enqueue(element: A): Unit =
_store.append(element)
def head: A =
_store.head
def dequeue: A =
_store.remove(0)
}
</code></pre>
<p class="fragment">What's the runtime?</p>
</section>
<section>
<p><b>Thought question:</b> How could you use an array to build a queue?</p>
</section>