553 lines
21 KiB
Plaintext
553 lines
21 KiB
Plaintext
---
|
||
template: templates/cse4562_2021_slides.erb
|
||
title: "Cost-Based Optimization"
|
||
date: March 9, 2021
|
||
textbook: Ch. 16
|
||
---
|
||
|
||
<!-- 2019 by OK
|
||
This went pretty well. If anything, it might be nice to adjust the \select \distinct propagation formula (uniform prior section) lists into animated tables for consistency with the rest of the presentation.
|
||
-->
|
||
|
||
<section>
|
||
<section>
|
||
<h3>General Query Optimizers</h3>
|
||
<ol style="font-size: 60%">
|
||
<li>Apply blind heuristics (e.g., push down selections)</li>
|
||
<li>Enumerate all possible <i>execution plans</i> by varying (or for a reasonable subset)
|
||
<ul>
|
||
<li>Join/Union Evaluation Order (commutativity, associativity, distributivity)</li>
|
||
<li>Algorithms for Joins, Aggregates, Sort, Distinct, and others</li>
|
||
<li>Data Access Paths</li>
|
||
</ul>
|
||
</li>
|
||
<li class="fragment highlight-blue">Estimate the cost of each execution plan</li>
|
||
<li>Pick the execution plan with the lowest cost</li>
|
||
</ol>
|
||
</section>
|
||
</section>
|
||
|
||
<section>
|
||
<section>
|
||
<p><b>Idea 1: </b> Run each plan</p>
|
||
</section>
|
||
|
||
<section>
|
||
<img src="graphics/Clipart/facepalm.jpg" class="stretch" />
|
||
<attribution>© Paramount Pictures</attribution>
|
||
</section>
|
||
|
||
<section>
|
||
<p>If we can't get the exact cost of a plan, what can we do?</p>
|
||
</section>
|
||
|
||
<section>
|
||
<p class="fragment highlight-grey"><b>Idea 2: </b> Run each plan on a small sample of the data.</p>
|
||
<p style="margin-top: 50px;"><b>Idea 3: </b> Analytically estimate the cost of a plan.</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Plan Cost</h3>
|
||
<dl>
|
||
<div class="fragment" data-fragment-index="1"><div class="fragment highlight-grey" data-fragment-index="4">
|
||
<dt>CPU Time</dt>
|
||
<dd>How much time is spent processing.</dd>
|
||
</div></div>
|
||
|
||
<div class="fragment" data-fragment-index="2">
|
||
<dt># of IOs</dt>
|
||
<dd>How many random reads + writes go to disk.</dd>
|
||
</div>
|
||
|
||
<div class="fragment" data-fragment-index="3">
|
||
<dt>Memory Required</dt>
|
||
<dd>How much memory do you need.</dd>
|
||
</div>
|
||
</dl>
|
||
</section>
|
||
|
||
<section>
|
||
<img src="2021-03-09/EstimationXKCD.png">
|
||
<attribution>Randal Munroe (<a href="https://creativecommons.org/licenses/by-nc/2.5/">cc-by-nc</a>)</attribution>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Remember the Real Goals</h3>
|
||
<ol>
|
||
<li class="fragment">Accurately <b>rank</b> the plans.</li>
|
||
<li class="fragment">Don't spend more time optimizing than you get back.</li>
|
||
<li class="fragment">Don't pick a plan that uses more memory than you have.</li>
|
||
</ol>
|
||
</section>
|
||
</section>
|
||
|
||
<!-- ============================================ -->
|
||
|
||
<section>
|
||
<section>
|
||
<h3>Accounting</h3>
|
||
<p class="fragment" data-fragment-index="1" style="margin-top: 50px;">Figure out the IO cost of the <b>entire</b><span class="fragment" data-fragment-index="2">*</span> subtree.</p>
|
||
|
||
<p class="fragment" style="margin-top: 50px;" data-fragment-index="3">Only count the amount of memory <b>added</b> by each operator.</p>
|
||
|
||
|
||
<p class="fragment" data-fragment-index="2" style="margin-top: 50px; font-size: 80%">* Different from earlier in the semester.</p>
|
||
|
||
</section>
|
||
|
||
<section>
|
||
<table style="font-size: 70%">
|
||
<tr><th>Operation</th><th>RA</th><th>Total IOs (#pages)</th><th>Memory (#tuples)</th></tr>
|
||
<tr class="fragment" data-fragment-index="0">
|
||
<td>Table Scan</td>
|
||
<td>$R$</td>
|
||
<td class="fragment" data-fragment-index="1">$\frac{|R|}{\mathcal P}$</td>
|
||
<td class="fragment" data-fragment-index="2">$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="3">
|
||
<td>Projection</td>
|
||
<td>$\pi(R)$</td>
|
||
<td class="fragment" data-fragment-index="4">$\textbf{io}(R)$</td>
|
||
<td class="fragment" data-fragment-index="4">$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="5">
|
||
<td>Selection</td>
|
||
<td>$\sigma(R)$</td>
|
||
<td>$\textbf{io}(R)$</td>
|
||
<td>$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="6">
|
||
<td>Union</td>
|
||
<td>$R \uplus S$</td>
|
||
<td>$\textbf{io}(R) + \textbf{io}(S)$</td>
|
||
<td>$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="7">
|
||
<td style="vertical-align: middle;">Sort <span class="fragment" data-fragment-index="8">(In-Mem)</span></td>
|
||
<td style="vertical-align: middle;">$\tau(R)$</td>
|
||
<td class="fragment" data-fragment-index="8">$\textbf{io}(R)$</td>
|
||
<td class="fragment" data-fragment-index="9">$O(|R|)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td class="fragment" data-fragment-index="10">Sort (On-Disk)</td>
|
||
<td class="fragment" data-fragment-index="10">$\tau(R)$</td>
|
||
<td class="fragment" data-fragment-index="11">$\frac{2 \cdot \lfloor log_{\mathcal B}(|R|) \rfloor}{\mathcal P} + \textbf{io}(R)$</td>
|
||
<td class="fragment" data-fragment-index="10">$O(\mathcal B)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="12">
|
||
<td><span class="fragment" data-fragment-index="13">(B+Tree)</span> Index Scan</td>
|
||
<td>$Index(R, c)$</td>
|
||
<td class="fragment" data-fragment-index="13">$\log_{\mathcal I}(|R|) + \frac{|\sigma_c(R)|}{\mathcal P}$</td>
|
||
<td class="fragment" data-fragment-index="14">$O(1)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td span class="fragment" data-fragment-index="15">(Hash) Index Scan</td>
|
||
<td span class="fragment" data-fragment-index="15">$Index(R, c)$</td>
|
||
<td class="fragment" data-fragment-index="15">$1$</td>
|
||
<td class="fragment" data-fragment-index="16">$O(1)$</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<ol style="font-size: 50%; margin-top: 50px;">
|
||
<li class="fragment" data-fragment-index="1">Tuples per Page ($\mathcal P$) <span>– Normally defined per-schema</span></li>
|
||
<li class="fragment" data-fragment-index="1">Size of $R$ ($|R|$)</li>
|
||
<li class="fragment" data-fragment-index="10">Pages of Buffer ($\mathcal B$)</li>
|
||
<li class="fragment" data-fragment-index="13">Keys per Index Page ($\mathcal I$)</li>
|
||
</ol>
|
||
</section>
|
||
<section>
|
||
<table style="font-size: 70%">
|
||
<tr><th width="300px">Operation</th><th>RA</th><th>Total IOs (#pages)</th><th style="font-size: 80%;">Mem (#tuples)</th></tr>
|
||
<tr class="fragment" data-fragment-index="1">
|
||
<td style="font-size: 60%">Nested Loop Join <span class="fragment" data-fragment-index="2">(Buffer $S$ in mem)</span></td>
|
||
<td>$R \times_{mem} S$</td>
|
||
<td class="fragment" data-fragment-index="2">$\textbf{io}(R)+\textbf{io}(S)$</td>
|
||
<td class="fragment" data-fragment-index="3">$O(|S|)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td class="fragment" data-fragment-index="4" style="font-size: 60%">Block NLJ (Buffer $S$ on disk)</td>
|
||
<td class="fragment" data-fragment-index="4">$R \times_{disk} S$</td>
|
||
<td class="fragment" data-fragment-index="5">$\frac{|R|}{\mathcal B} \cdot \frac{|S|}{\mathcal P} + \textbf{io}(R) + \textbf{io}(S)$</td>
|
||
<td class="fragment" data-fragment-index="4">$O(1)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td class="fragment" data-fragment-index="4" style="font-size: 60%">Block NLJ (Recompute $S$)</td>
|
||
<td class="fragment" data-fragment-index="4">$R \times_{redo} S$</td>
|
||
<td class="fragment" data-fragment-index="6">$\textbf{io}(R) + \frac{|R|}{\mathcal B} \cdot \textbf{io}(S)$</td>
|
||
<td class="fragment" data-fragment-index="4">$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="7">
|
||
<td>1-Pass Hash Join</td>
|
||
<td>$R \bowtie_{1PH, c} S$</td>
|
||
<td class="fragment" data-fragment-index="8">$\textbf{io}(R) + \textbf{io}(S)$</td>
|
||
<td class="fragment" data-fragment-index="8">$O(|S|)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="9">
|
||
<td>2-Pass Hash Join</td>
|
||
<td>$R \bowtie_{2PH, c} S$</td>
|
||
<td class="fragment" data-fragment-index="10">$\frac{2|R| + 2|S|}{\mathcal P} + \textbf{io}(R) + \textbf{io}(S)$</td>
|
||
<td class="fragment" data-fragment-index="10">$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="11">
|
||
<td>Sort-Merge Join </td>
|
||
<td>$R \bowtie_{SM, c} S$</td>
|
||
<td class="fragment" data-fragment-index="12">[Sort]</td>
|
||
<td class="fragment" data-fragment-index="12">[Sort]</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="13">
|
||
<td><span class="fragment" data-fragment-index="14">(Tree)</span> Index NLJ</td>
|
||
<td>$R \bowtie_{INL, c}$</td>
|
||
<td class="fragment" data-fragment-index="14">$|R| \cdot (\log_{\mathcal I}(|S|) + \frac{|\sigma_c(S)|}{\mathcal P})$</td>
|
||
<td class="fragment" data-fragment-index="15">$O(1)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td class="fragment" data-fragment-index="16">(Hash) Index NLJ</td>
|
||
<td class="fragment" data-fragment-index="16">$R \bowtie_{INL, c}$</td>
|
||
<td class="fragment" data-fragment-index="16">$|R| \cdot 1$</td>
|
||
<td class="fragment" data-fragment-index="17">$O(1)$</td>
|
||
</tr>
|
||
<tr class="fragment" data-fragment-index="18">
|
||
<td><span class="fragment" data-fragment-index="19">(In-Mem)</span> Aggregate</td>
|
||
<td>$\gamma_A(R)$</td>
|
||
<td class="fragment" data-fragment-index="19">$\textbf{io}(R)$</td>
|
||
<td class="fragment" data-fragment-index="20">$adom(A)$</td>
|
||
</tr>
|
||
<tr>
|
||
<td class="fragment" data-fragment-index="21" style="font-size: 90%">(Sort/Merge) Aggregate</td>
|
||
<td class="fragment" data-fragment-index="21">$\gamma_A(R)$</td>
|
||
<td class="fragment" data-fragment-index="21">[Sort]</td>
|
||
<td class="fragment" data-fragment-index="21">[Sort]</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<ol style="font-size: 50%;">
|
||
<li>Tuples per Page ($\mathcal P$) <span>– Normally defined per-schema</span></li>
|
||
<li>Size of $R$ ($|R|$)</li>
|
||
<li>Pages of Buffer ($\mathcal B$)</li>
|
||
<li>Keys per Index Page ($\mathcal I$)</li>
|
||
<li class="fragment" data-fragment-index="20">Number of distinct values of $A$ ($adom(A)$)</li>
|
||
</ol>
|
||
</section>
|
||
|
||
<section>
|
||
<table style="font-size: 70%">
|
||
<tr><th>Symbol</th><th>Parameter</th><th>Type</th></th></tr>
|
||
<tr>
|
||
<td>$\mathcal P$</td><td>Tuples Per Page</td>
|
||
<td class="fragment" data-fragment-index="1">Fixed ($\frac{|\text{page}|}{|\text{tuple}|}$)</td>
|
||
</tr>
|
||
<tr>
|
||
<td>$|R|$</td><td>Size of $R$</td>
|
||
<td class="fragment" data-fragment-index="2">Precomputed<span class="fragment" data-fragment-index="6">$^*$</span> ($|R|$)</td>
|
||
</tr>
|
||
<tr>
|
||
<td>$\mathcal B$</td><td>Pages of Buffer</td>
|
||
<td class="fragment" data-fragment-index="3">Configurable Parameter</td>
|
||
</tr>
|
||
<tr>
|
||
<td>$\mathcal I$</td><td>Keys per Index Page</td>
|
||
<td class="fragment" data-fragment-index="4">Fixed ($\frac{|\text{page}|}{|\text{key+pointer}|}$)</td>
|
||
</tr>
|
||
<tr>
|
||
<td>$adom(A)$</td><td>Number of distinct values of $A$</td>
|
||
<td class="fragment" data-fragment-index="5">Precomputed<span class="fragment" data-fragment-index="6">$^*$</span> ($|\delta_A(R)|$)</td>
|
||
</tr>
|
||
</table>
|
||
<p class="fragment" data-fragment-index="6" style="font-size: 50%">* unless $R$ is a query</p>
|
||
</section>
|
||
|
||
</section>
|
||
<!-- ============================================ -->
|
||
|
||
<section>
|
||
<section>
|
||
<p>Estimating IOs requires Estimating $|Q(R)|$, $|\delta_A(Q(R))|$</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Cardinality Estimation</h3>
|
||
<p class="fragment">Unlike estimating IOs, cardinality estimation doesn't care about the algorithm, so we'll just be working with raw RA.</p>
|
||
</section>
|
||
|
||
<section>
|
||
<table style="font-size: 70%">
|
||
<tr>
|
||
<th>Operator</th>
|
||
<th>RA</th>
|
||
<th>Estimated Size</th>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Table</td>
|
||
<td>$R$</td>
|
||
<td class="fragment" data-fragment-index="1">$|R|$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Projection</td>
|
||
<td>$\pi(Q)$</td>
|
||
<td class="fragment" data-fragment-index="2">$|Q|$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Union</td>
|
||
<td>$Q_1 \uplus Q_2$</td>
|
||
<td class="fragment" data-fragment-index="3">$|Q_1| + |Q_2|$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Cross Product</td>
|
||
<td>$Q_1 \times Q_2$</td>
|
||
<td class="fragment" data-fragment-index="4">$|Q_1| \times |Q_2|$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Sort</td>
|
||
<td>$\tau(Q)$</td>
|
||
<td class="fragment" data-fragment-index="5">$|Q|$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Limit</td>
|
||
<td>$\texttt{LIMIT}_N(Q)$</td>
|
||
<td class="fragment" data-fragment-index="6">$N$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Selection</td>
|
||
<td>$\sigma_c(Q)$</td>
|
||
<td class="fragment" data-fragment-index="8">$|Q| \times \texttt{SEL}(c, Q)$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Join</td>
|
||
<td>$Q_1 \bowtie_c Q_2$</td>
|
||
<td class="fragment" data-fragment-index="9">$|Q_1| \times |Q_2| \times \texttt{SEL}(c, Q_1\times Q_2)$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Distinct</td>
|
||
<td>$\delta_A(Q)$</td>
|
||
<td class="fragment" data-fragment-index="11">$\texttt{UNIQ}(A, Q)$</td>
|
||
</tr>
|
||
|
||
<tr>
|
||
<td>Aggregate</td>
|
||
<td>$\gamma_{A, B \leftarrow \Sigma}(Q)$</td>
|
||
<td class="fragment" data-fragment-index="12">$\texttt{UNIQ}(A, Q)$</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<ul style="font-size: 50%; margin-top: 20px">
|
||
<li class="fragment" data-fragment-index="7">$\texttt{SEL}(c, Q)$: Selectivity of $c$ on $Q$, or $\frac{|\sigma_c(Q)|}{|Q|}$</li>
|
||
<li class="fragment" data-fragment-index="10">$\texttt{UNIQ}(A, Q)$: # of distinct values of $A$ in $Q$.</li>
|
||
</ul>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Cardinality Estimation</h3>
|
||
<h4>(The Hard Parts)</h4>
|
||
|
||
<dl>
|
||
<dt style="margin-top: 50px;">$\sigma_c(Q)$ (Cardinality Estimation)</dt>
|
||
<dd>How many tuples will a condition $c$ allow to pass?</dd>
|
||
|
||
<dt style="margin-top: 50px;">$\delta_A(Q)$ (Distinct Values Estimation)</dt>
|
||
<dd>How many distinct values of attribute(s) $A$ exist?</dd>
|
||
</dl>
|
||
</section>
|
||
</section>
|
||
|
||
<section>
|
||
<section>
|
||
<p><b>Idea 1:</b> Assume each selection filters down to 10% of the data.</p>
|
||
</section>
|
||
|
||
<section>
|
||
<img src="graphics/Clipart/facepalm.jpg" class="stretch" />
|
||
<p class="fragment">no... really!</p>
|
||
<attribution>© Paramount Pictures</attribution>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>... there are problems</h3>
|
||
<div class="fragment">
|
||
<h4>Inconsistent estimation</h4>
|
||
<p style="font-size:70%;">$|\sigma_{c_1}(\sigma_{c_2}(R))| \neq |\sigma_{c_1 \wedge c_2}(R)|$</p>
|
||
</div>
|
||
<div class="fragment">
|
||
<h4>Too consistent estimation</h4>
|
||
<p style="font-size:70%;">$|\sigma_{id = 1}(\texttt{STUDENTS})| = |\sigma_{residence = 'NY'}(\texttt{STUDENTS})|$</p>
|
||
</div>
|
||
<p style="margin-top: 100px" class="fragment">... but remember that all we need is to <u>rank</u> plans.</p>
|
||
</section>
|
||
|
||
<section>
|
||
<p>Many major databases (Oracle, Postgres, Teradata, etc...) use something like 10% rule if they have nothing better.</p>
|
||
|
||
|
||
<p class="fragment" style="font-size: 80%; margin-top: 20px;">(The specific % varies by DBMS.)</p>
|
||
|
||
<p class="fragment" style="font-size: 80%; margin-top: 20px;">(Teradata uses 10% for the first <code>AND</code> clause,<br/>cut by another 75% for every subsequent clause)</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>(Some) Estimation Techniques</h3>
|
||
|
||
<dl style="font-size: 80%">
|
||
<div>
|
||
<dt>The 10% rule</dt>
|
||
<dd>Rules of thumb if you have no other options...</dd>
|
||
</div>
|
||
|
||
<div class="fragment">
|
||
<dt>Uniform Prior</dt>
|
||
<dd>Use basic statistics to make a very rough guess.</dd>
|
||
</div>
|
||
|
||
<div class="fragment">
|
||
<dt>Sampling / History</dt>
|
||
<dd>Small, Quick Sampling Runs (or prior executions of the query).</dd>
|
||
</div>
|
||
|
||
<div class="fragment">
|
||
<dt>Histograms</dt>
|
||
<dd>Using more detailed statistics for improved guesses.</dd>
|
||
</div>
|
||
|
||
<div class="fragment">
|
||
<dt>Constraints</dt>
|
||
<dd>Using rules about the data for improved guesses.</dd>
|
||
</div>
|
||
</dl>
|
||
</section>
|
||
</section>
|
||
|
||
<!-- ============================================ -->
|
||
|
||
<section>
|
||
|
||
<section>
|
||
<h3>Uniform Prior</h3>
|
||
|
||
<p style="text-align: left; margin-bottom: 0px; font-weight: bold;">We assume that for $\sigma_c(Q)$ or $\delta_A(Q)$...</p>
|
||
<ol>
|
||
<li>Basic statistics are known about $Q$: <ul>
|
||
<li style="margin-top: 0px;"><code>COUNT(*)</code></li>
|
||
<li style="margin-top: 0px;"><code>COUNT(DISTINCT A)</code> (for each A)</li>
|
||
<li style="margin-top: 0px;"><code>MIN(A)</code>, <code>MAX(A)</code> (for each numeric A)</li>
|
||
</ul></li>
|
||
<li>Attribute values are uniformly distributed.</li>
|
||
<li>No inter-attribute correlations.</li>
|
||
</ol>
|
||
<p class="fragment" style="font-size: 80%; font-weight: bold; margin-top: 20px;">
|
||
If necessary statistics aren't available (point 1), fall back to the 10% rule.
|
||
</p>
|
||
<p class="fragment" style="font-size: 80%; font-weight: bold; margin-top: 20px;">
|
||
If statistical assumptions (points 2, 3) aren't perfectly true, we'll still likely be getting a better estimate than the 10% rule.
|
||
</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>COUNT(DISTINCT A)</h3>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$\texttt{UNIQ}(A, \pi_{A, \ldots}(R)) = \texttt{UNIQ}(A, R)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$\texttt{UNIQ}(A, \sigma(R)) \approx \texttt{UNIQ}(A, R)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$\texttt{UNIQ}(A, R \times S) = \texttt{UNIQ}(A, R)$ or $\texttt{UNIQ}(A, S)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$$max(\texttt{UNIQ}(A, R), \texttt{UNIQ}(A, S)) \leq\\ \texttt{UNIQ}(A, R \uplus S)\\ \leq \texttt{UNIQ}(A, R) + \texttt{UNIQ}(A, S)$$</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>MIN(A), MAX(A)</h3>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$min_A(\pi_{A, \ldots}(R)) = min_A(R)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$min_A(\sigma_{A, \ldots}(R)) \approx min_A(R)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$min_A(R \times S) = min_A(R)$ or $min_A(S)$</p>
|
||
<p class="fragment" style="font-size: 70%; margin-top: 50px;">$min_A(R \uplus S) = min(min_A(R), min_A(S))$</p>
|
||
</section>
|
||
|
||
<section>
|
||
<p>Estimating $\delta_A(Q)$ requires only <code>COUNT(DISTINCT A)</code></p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Estimating Selectivity</h3>
|
||
|
||
<p>Selectivity is a probability ($\texttt{SEL}(c, Q) = P(c)$)</p>
|
||
<table style="font-size: 85%">
|
||
<tr class="fragment">
|
||
<td>$P(A = x_1)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment">$\frac{1}{\texttt{COUNT(DISTINCT A)}}$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(A \in (x_1, x_2, \ldots, x_N))$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment">$\frac{N}{\texttt{COUNT(DISTINCT A)}}$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(A \leq x_1)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment">$\frac{x_1 - \texttt{MIN(A)}}{\texttt{MAX(A)} - \texttt{MIN(A)}}$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(x_1 \leq A \leq x_2)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment">$\frac{x_2 - x_1}{\texttt{MAX(A)} - \texttt{MIN(A)}}$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(A = B)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment" style="font-size: 60%">$\textbf{min}\left( \frac{1}{\texttt{COUNT(DISTINCT A)}}, \frac{1}{\texttt{COUNT(DISTINCT B)}} \right)$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(c_1 \wedge c_2)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment" >$P(c_1) \cdot P(c_2)$</td>
|
||
</tr>
|
||
|
||
<tr class="fragment">
|
||
<td>$P(c_1 \vee c_2)$</td>
|
||
<td>$=$</td>
|
||
<td class="fragment" >$1 - (1 - P(c_1)) \cdot (1 - P(c_2))$</td>
|
||
</tr>
|
||
</table>
|
||
|
||
<p style="font-size: 60%">(With constants $x_1$, $x_2$, ...)</p>
|
||
</section>
|
||
|
||
<section>
|
||
<h3>Limitations</h3>
|
||
|
||
<dl>
|
||
<div>
|
||
<dt>Don't always have statistics for $Q$</dt>
|
||
<dd>For example, $\pi_{A \leftarrow (B \cdot C)}(R)$</dd>
|
||
</div>
|
||
|
||
<div>
|
||
<dt>Don't always have clear rules for $c$</dt>
|
||
<dd>For example, $\sigma_{\texttt{FitsModel}(A, B, C)}(R)$</dd>
|
||
</div>
|
||
|
||
<div>
|
||
<dt>Attribute values are not always uniformly distributed.</dt>
|
||
<dd>For example, <span style="font-size: 60%"> $|\sigma_{SPC\_COMMON = 'pin\ oak'}(T)|$ vs $|\sigma_{SPC\_COMMON = 'honeylocust'}(T)|$</span></dd>
|
||
</div>
|
||
|
||
<div>
|
||
<dt>Attribute values are sometimes correlated.</dt>
|
||
<dd>For example, $\sigma_{(stump < 5) \wedge (diam > 3)}(T)$</dd>
|
||
</div>
|
||
</dl>
|
||
<p class="fragment">...but handles <b>most</b> usage patterns</p>
|
||
</section>
|
||
|
||
<section>
|
||
... next class more!
|
||
</section>
|
||
|
||
</section> |