Recursion and average slides
parent
47de5ad22b
commit
fd13b7ca25
|
@ -47,313 +47,376 @@ textbook: Ch. 15
|
|||
</section>
|
||||
|
||||
<section>
|
||||
<p>All of the "work" is in the combine step.</p>
|
||||
<p>All of the "work" is in the combine step.</p>
|
||||
|
||||
<p class="fragment">Can we put the work in the divide step?</p>
|
||||
<p class="fragment">Can we put the work in the divide step?</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Idea 1</b>: Partition the data on the median value.</p>
|
||||
<p class="fragment"><b>Idea 2</b>: Partition the data in-place.</p>
|
||||
<p><b>Idea 1</b>: Partition the data on the median value.</p>
|
||||
<p class="fragment"><b>Idea 2</b>: Partition the data in-place.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<svg data-src="graphics/12b/quicksort-ideal.svg" />
|
||||
<svg data-src="graphics/12b/quicksort-ideal.svg" />
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">QuickSort (Idealized) <span class="fragment" data-fragment-index="2">(Wrong)</span></h4>
|
||||
<h4 class="slide_title">QuickSort (Idealized) <span class="fragment" data-fragment-index="2">(Wrong)</span></h4>
|
||||
|
||||
To sort an array of size $n$:
|
||||
<ol>
|
||||
<li class="fragment highlight-red" data-fragment-index="1">Pick a $pivot$ value.</li>
|
||||
<li>Swap values until...<ul>
|
||||
<li>array elements at $[1, \frac{n}{2})$ are $\leq pivot$</li>
|
||||
<li>array elements at $[\frac{n}{2}, n)$ are $> pivot$</li>
|
||||
</ul></li>
|
||||
<li>Recursively sort $low$</li>
|
||||
<li>Recursively sort $high$</li>
|
||||
</ol>
|
||||
To sort an array of size $n$:
|
||||
<ol>
|
||||
<li class="fragment highlight-red" data-fragment-index="1">Pick a $pivot$ value.</li>
|
||||
<li>Swap values until...<ul>
|
||||
<li>array elements at $[1, \frac{n}{2})$ are $\leq pivot$</li>
|
||||
<li>array elements at $[\frac{n}{2}, n)$ are $> pivot$</li>
|
||||
</ul></li>
|
||||
<li>Recursively sort $low$</li>
|
||||
<li>Recursively sort $high$</li>
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">QuickSort (Wrong)</h4>
|
||||
|
||||
<pre><code class="scala">
|
||||
def idealizedQuickSort(arr: Array[Int], from: Int, until: Int): Unit =
|
||||
{
|
||||
if(until - from < 1){ return }
|
||||
val pivot = ???
|
||||
var low = from, high = until -1
|
||||
<h4 class="slide_title">QuickSort (Wrong)</h4>
|
||||
|
||||
<pre><code class="scala">def idealizedQuickSort(arr: Array[Int], from: Int, until: Int): Unit =
|
||||
{
|
||||
if(until - from < 1){ return }
|
||||
val pivot = ???
|
||||
var low = from, high = until -1
|
||||
|
||||
while(low < high){
|
||||
while(arr(low) <= pivot && low < high){ low ++ }
|
||||
if(low < high){
|
||||
while(arr(high) > pivot && low < high){ high ++ }
|
||||
swap(arr, low, high)
|
||||
}
|
||||
}
|
||||
idealizedQuickSort(arr, from = 0, until = low)
|
||||
idealizedQuickSort(arr, from = low, until = until)
|
||||
}
|
||||
</code></pre>
|
||||
while(low < high){
|
||||
while(arr(low) <= pivot && low < high){ low ++ }
|
||||
if(low < high){
|
||||
while(arr(high) > pivot && low < high){ high ++ }
|
||||
swap(arr, low, high)
|
||||
}
|
||||
}
|
||||
idealizedQuickSort(arr, from = 0, until = low)
|
||||
idealizedQuickSort(arr, from = low, until = until)
|
||||
}</code></pre>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">Partition (Ideal)</h4>
|
||||
<svg data-src="graphics/12b/pivot-ideal.svg"/>
|
||||
<h4 class="slide_title">Partition (Ideal)</h4>
|
||||
<svg data-src="graphics/12b/pivot-ideal.svg"/>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">QuickSort (Wrong)</h4>
|
||||
<p>If we can obtain a pivot in $O(1)$, what's the complexity?</p>
|
||||
<h4 class="slide_title">QuickSort (Wrong)</h4>
|
||||
<p>If we can obtain a pivot in $O(1)$, what's the complexity?</p>
|
||||
|
||||
<p class="fragment">
|
||||
$$T_{quicksort}(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n = 1\\
|
||||
2 \cdog \T(\frac{n}{2}) + \Theta(n) + 0 & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</p>
|
||||
<p class="fragment">
|
||||
$$T_{quicksort}(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n = 1\\
|
||||
2 \cdot T(\frac{n}{2}) + \Theta(n) + 0 & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</p>
|
||||
|
||||
<p class="fragment">
|
||||
Contrast with <tt>MergeSort</tt>:
|
||||
$$T_{mergesort}(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n = 1\\
|
||||
2 \cdog \T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</p>
|
||||
<p class="fragment">
|
||||
Contrast with <tt>MergeSort</tt>:
|
||||
$$T_{mergesort}(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n = 1\\
|
||||
2 \cdot T(\frac{n}{2}) + \Theta(1) + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<p><b>Problem: </b> Finding the median value of an unsorted collection is $O(n\log(n))$</p>
|
||||
|
||||
<p class="fragment">(We'll talk about heaps later)</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<p><b>Idea: </b> If we pick a value at random, on average half the values will be lower.</p>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<p><b>Idea: </b> If we pick a value at random, <br>on average half the values will be lower.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
Let's say we pick the $i$th largest element, the recursive runtime for $T(n)$ is:
|
||||
$$\begin{cases}
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } i = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } i = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } i = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } i = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } i = n\\
|
||||
\end{cases}$$
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
|
||||
<ol>
|
||||
<li>Pick a value at random as a $pivot$.</li>
|
||||
<li>Swap values until the array is subdivided into...<ul>
|
||||
<li>$low$: array elements that are $\leq pivot$</li>
|
||||
<li>$pivot$</li>
|
||||
<li>$high$: array elements that are $> pivot$</li>
|
||||
</ul></li>
|
||||
<li>Recursively sort $low$</li>
|
||||
<li>Recursively sort $high$</li>
|
||||
</ol>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>How likely are we to pick $i = k$ for any specific $k$?</p>
|
||||
|
||||
<p class="fragment">$P[i = k] = \frac{1}{n}</p>
|
||||
<p>What's the worst-case runtime?</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>... a brief aside...</p>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<p>What if we always pick the worst pivot?</p>
|
||||
|
||||
<p class="fragment">[8, 7, 6, 5, 4, 3, 2, 1]</p>
|
||||
<p class="fragment">[7, 6, 5, 4, 3, 2, 1], 8, []</p>
|
||||
<p class="fragment">[6, 5, 4, 3, 2, 1], 7, [], 8</p>
|
||||
<p class="fragment">[5, 4, 3, 2, 1], 6, [], 7, 8</p>
|
||||
<p class="fragment">...</p>
|
||||
|
||||
<p class="fragment">$$T_{quicksort}(n) \in O(n^2)$$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">Probabilities and Expectations</h4>
|
||||
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $k$ times, what is the average over all possible outcomes?</p>
|
||||
<p>Is the worst case runtime representative?</p>
|
||||
<p class="fragment"><b>No!</b> (it'll almost always be faster)</p>
|
||||
<p class="fragment">Is there something we <b>can</b> say about the runtime?</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">k = 1</h4>
|
||||
<h4 class="slide_title">QuickSort</h4>
|
||||
<p>
|
||||
Let's say we pick the $X$th largest element as pivot,<br>What's the recursive runtime for $T(n)$?
|
||||
</p>
|
||||
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $1$ time, what is the average over all possible outcomes?</p>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Roll</th>
|
||||
<th>Probability</th>
|
||||
<th>Contribution</th>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td>⚀</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚁</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>2</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚂</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>3</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚃</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚄</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>5</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚅</td>
|
||||
<td>$\frac{1}{6}</td>
|
||||
<td>6</td>
|
||||
</tr>
|
||||
|
||||
$$\frac{1 + 2 + 3 + 4 + 5 + 6}{6} = \frac{1}{6}\cdot 1 + \frac{1}{6}\cdot 2 + \frac{1}{6}\cdot 3 + \frac{1}{6}\cdot 4 + \frac{1}{6}\cdot 5 + \frac{1}{6}\cdot 6 = \sum_{i} \texttt{Probability}_i \cdot \texttt{Contribution}_i = 3.5$$
|
||||
</table>
|
||||
<p class="fragment">
|
||||
$$\begin{cases}
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } X = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } X = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } X = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } X = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } X = n\\
|
||||
\end{cases}$$
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>If $X$ is a random variable representing the outcome of the roll, we call this the <i>expectation</i> of $X$, or $E[X]$</p>
|
||||
<p>How likely are we to pick $X = k$ for any specific $k$?</p>
|
||||
|
||||
<p class="fragment">$P[X = k] = \frac{1}{n}$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">k = 2</h4>
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $2$ times, what is the average over all possible outcomes?</p>
|
||||
|
||||
<p>Does the outcome of one roll affect the other?</p>
|
||||
<p class="fragment"><b>No:</b> Each roll is an <i>independent event</i>.</p>
|
||||
<p class="fragment">If $X$ and $Y$ are random variables representing the outcome of each roll (i.e., independent random variables), $E[X + Y] = E[X] + E[Y] = 3.5 + 3.5 = 7$</p>
|
||||
<p>... a brief aside...</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">Expectations</h4>
|
||||
|
||||
$$E_i[X] = \sum_{i} P_i \cdot X_i$$
|
||||
<h4 class="slide_title">Probabilities and Expectations</h4>
|
||||
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $k$ times,<br>what is the average over all possible outcomes?</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">k = 1</h4>
|
||||
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $1$ time...</p>
|
||||
|
||||
<table style="font-size: 70%">
|
||||
<tr>
|
||||
<th>Roll</th>
|
||||
<th>Probability</th>
|
||||
<th>Contribution</th>
|
||||
</tr>
|
||||
|
||||
<tr>
|
||||
<td>⚀</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚁</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>2</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚂</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>3</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚃</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚄</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>5</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>⚅</td>
|
||||
<td>$\frac{1}{6}$</td>
|
||||
<td>6</td>
|
||||
</tr>
|
||||
</table>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">k = 1</h4>
|
||||
|
||||
<div>
|
||||
$$\frac{1 + 2 + 3 + 4 + 5 + 6}{6}= 3.5$$
|
||||
</div>
|
||||
<div class="fragment" style="margin-top: 50px;">
|
||||
$$ = \frac{1}{6}\cdot 1 + \frac{1}{6}\cdot 2 + \frac{1}{6}\cdot 3 + \frac{1}{6}\cdot 4 + \frac{1}{6}\cdot 5 + \frac{1}{6}\cdot 6$$
|
||||
</div>
|
||||
<div class="fragment" style="margin-top: 50px;">
|
||||
$$= \sum_{i} \texttt{Probability}_i \cdot \texttt{Contribution}_i$$
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>If $X$ is a random variable representing the outcome of the roll, we call this the <i>expectation</i> of $X$, or $E[X]$</p>
|
||||
|
||||
<p class="fragment">
|
||||
$$E[X] = \sum_{i} P_i \cdot X_i$$
|
||||
</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">k = 2</h4>
|
||||
<p>If I roll d6 (a 6-sided die 🎲) $2$ times...</p>
|
||||
|
||||
<p>Does the outcome of one roll affect the other?</p>
|
||||
<p class="fragment"><b>No:</b> Each roll is an <i>independent event</i>.</p>
|
||||
<p class="fragment">If $X$ and $Y$ are random variables representing the outcome of each roll (i.e., independent random variables), $E[X + Y] = E[X] + E[Y]$</p>
|
||||
<p class="fragment">$= 3.5 + 3.5 = 7$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$T(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } n > 1 \wedge X = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } n > 1 \wedge X = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } n > 1 \wedge X = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } n > 1 \wedge X = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } n > 1 \wedge X = n\\
|
||||
\end{cases}$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>We pick the $X$th largest element as a pivot</p>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
E[T(X-1) + T(n-X)] + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$\begin{cases}
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } X = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } X = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } X = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } X = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } X = n\\
|
||||
\end{cases}$$
|
||||
<p>There's a symmetry: The left varies from $T(0)$ to $T(n-1)$, and visa versa on the right.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>We pick the $X$th largest element as a pivot</p>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
E[2 \cdot T(X-1)] + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
<p> </p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>We pick the $X$th largest element as a pivot</p>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
2 E[T(X-1)] + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
<p> </p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p> </p>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
2 E\left[\sum_{i=0}^{n-1} \frac{1}{n} T(i) \right] + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
<p class="fragment">Each $T(X-1)$ is <i>independent</i>.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>Back to induction...</p>
|
||||
|
||||
<p><b>Hypothesis: </b> $E[T(n)] \in O(n \log(n))$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Base Case: </b> $E[T(1)] \lec c(1 \log(1))$</p>
|
||||
|
||||
<p class="fragment">$$E[T(1)] \leq c \cdot (1 \log(1))$$</p>
|
||||
<p class="fragment">$$E[T(1)] \leq c \cdot (1 \cdot 0)$$</p>
|
||||
<p class="fragment">$$E[T(1)] \not \leq 0$$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Base Case (take Two): </b> $E[T(2)] \leq c(2 \log(2))$</p>
|
||||
|
||||
<p class="fragment">$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$</p>
|
||||
<p class="fragment">$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$</p>
|
||||
<p class="fragment">$$T(0) + T(1) + 2c_1 \leq 2c$$</p>
|
||||
<p class="fragment">$$2c_0 + 2c_1 \leq 2c$$</p>
|
||||
<p class="fragment">True for any $c \geq c_0 + c_1$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Assume: </b> $E[T(n')] \leq c(n' \log(n'))$ for <b>all</b> $n' < n$</p>
|
||||
<p><b>Show: </b> $E[T(n)] \leq c(n \log(n))$</p>
|
||||
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$\frac{2}{n}\left(\sum_{i=0}^{n-1} c i \log(i) \right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$</p>
|
||||
<p class="fragment">$$c_1 \leq c\log(n)$$</p>
|
||||
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$T(n) = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } i = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } i = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } i = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } i = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } i = n\\
|
||||
\end{cases}$$
|
||||
<p>$E[T_{quicksort}(n)] = O(n\log(n))$</p>
|
||||
|
||||
<p>So is Quicksort $O(n\log(n))$? <span class="fragment"><b>No!</b></span></p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>$i$ is the random variable</p>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
E[T(i-1) + T(n-i)] + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
<h4 class="slide_title">What <b>guarantees</b> do you get?</h4>
|
||||
<dl>
|
||||
<dt>$f(n)$ is a Tight Bound</dt>
|
||||
<dd>The algorithm <b>always</b> runs in $cf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is a Worst-Case Bound</dt>
|
||||
<dd>The algorithm <b>always</b> runs at-most $cf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is an Amortized Worst-Case Bound</dt>
|
||||
<dd>$n$ invocations of the algorithm <b>always</b> run in $cnf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is an Average Bound</dt>
|
||||
<dd>🤷</dd>
|
||||
</dl>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$\begin{cases}
|
||||
T(0) + T(n-1) + \Theta(n) & \textbf{if } i = 1\\
|
||||
T(1) + T(n-2) + \Theta(n) & \textbf{if } i = 2\\
|
||||
T(2) + T(n-3) + \Theta(n) & \textbf{if } i = 3\\
|
||||
..\\
|
||||
T(n-2) + T(1) + \Theta(n) & \textbf{if } i = n-1\\
|
||||
T(n-1) + T(0) + \Theta(n) & \textbf{if } i = n\\
|
||||
\end{cases}$$
|
||||
<p>There's a symmetry: The left varies from $T(0)$ to $T(n-1)$, and visa versa right.</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
\sum_{i=0}^{n-1}\left( \frac{1}{n} 2\cdot E[T(i)] \right) + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
$$E[T(n)] = \begin{cases}
|
||||
\Theta(1) & \textbf{if } n \leq 1\\
|
||||
\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + \Theta(n) & \textbf{otherwise}
|
||||
\end{cases}$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>Back to induction...</p>
|
||||
|
||||
<p><b>Hypothesis: </b> $E[T(n)] \in O(n \log(n))$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Base Case: </b> $E[T(1)] \in O(1 \log(1))$</p>
|
||||
|
||||
$$E[T(1)] \leq c \cdot (1 \log(1))$$
|
||||
$$E[T(1)] \leq c \cdot (1 \cdot 0)$$
|
||||
$$E[T(1)] \not \leq 0$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Base Case (take Two): </b> $E[T(2)] \in O(2 \log(2))$</p>
|
||||
|
||||
$$E[T(2)] \leq c \cdot (2 \log(2))$$
|
||||
$$2\cdot E_i[T(i-1)] + 2c_1 \leq 2c$$
|
||||
$$2\cdot \left(\frac{1}{2}T(0) + \frac{1}{2}T(1)\right) + 2c_1 \leq 2c$$
|
||||
$$T(0) + T(1) + 2c_1 \leq 2c$$
|
||||
$$2c_0 + 2c_1 \leq 2c$$
|
||||
<p>True for any $c \geq c_0 + c_1$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p><b>Assume: </b> $E[T(n')] \leq c(n' \log(n'))$ for <b>all</b> $n' < n$</p>
|
||||
<p><b>Show: </b> $E[T(n)] \leq c(n \log(n))$</p>
|
||||
$$E[T(n)] \leq c \cdot (n \log(n))$$
|
||||
$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} E[T(i)] \right) + c_1 \leq c n \log(n)$$
|
||||
$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(i) \right) + c_1 \leq c n \log(n)$$
|
||||
$$c\frac{2}{n}\left(\sum_{i=0}^{n-1} i \log(n) \right) + c_1 \leq c n \log(n)$$
|
||||
$$c\frac{2 \log(n)}{n}\left(\sum_{i=0}^{n-1} i \right) + c_1 \leq c n \log(n)$$
|
||||
$$c\frac{2 \log(n)}{n}\left( \frac{(n-1)(n-1+1)}{2}\right) + c_1 \leq c n \log(n)$$
|
||||
$$c\frac{\log(n)}{n}\left(n^2 - n\right) + c_1 \leq c n \log(n)$$
|
||||
$$cn\log(n) - c\log(n) + c_1 \leq c n \log(n)$$
|
||||
$$c_1 \leq c\log(n)$$
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>$E[T_{quicksort}(n)] = O(n\log(n))$</p>
|
||||
|
||||
<p>So is Quicksort $O(n\log(n))$? <span class="fragment"><b>No!</b></span></p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<p>What if we always pick the worst pivot?</p>
|
||||
|
||||
<p>[8, 7, 6, 5, 4, 3, 2, 1]</p>
|
||||
<p>[7, 6, 5, 4, 3, 2, 1], 8, []</p>
|
||||
<p>[6, 5, 4, 3, 2, 1], 7, [], 8</p>
|
||||
<p>[5, 4, 3, 2, 1], 6, [], 7, 8</p>
|
||||
<p>...</p>
|
||||
|
||||
<p>$$T_{quicksort}(n) \in O(n^2)$$</p>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">What guarantees do you get?</h4>
|
||||
<dl>
|
||||
<dt>$f(n)$ is a Tight Bound</dt>
|
||||
<dd>The algorithm <b>always</b> runs in $cf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is a Worst-Case Bound</dt>
|
||||
<dd>The algorithm <b>always</b> runs at-most $cf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is an Amortized Worst-Case Bound</dt>
|
||||
<dd>$n$ invocations of the algorithm <b>always</b> run in $cnf(n)$ steps</dd>
|
||||
|
||||
<dt>$f(n)$ is an Average Bound</dt>
|
||||
<dd>🤷</dd>
|
||||
</dl>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h4 class="slide_title">Next time...</h4>
|
||||
|
||||
<p>Special <tt>Seq</tt>uences: <tt>Stack</tt>, <tt>Queue</tt></p>
|
||||
<h4 class="slide_title">Next time...</h4>
|
||||
|
||||
<p>Special <tt>Seq</tt>uences: <tt>Stack</tt>, <tt>Queue</tt></p>
|
||||
</section>
|
||||
|
||||
|
|
Loading…
Reference in New Issue