First draft of slides

slides/talks/2017-1-EDBT-Inference/index.html

@@ -79,42 +79,6 @@
 			</section>
 		</section>
 
-		<section>
-			<section>
-				<h2>Online Aggregation (OLA)</h2>
-
-				<p style="margin-top: 60px;">$Avg(3,6,10,9,1,3,9,7,9,4,7,9,2,1,2,4,10,8,9,7) = 6$</p>
-				<p class="fragment">$Avg(3,6,10,9,1) = 5.8$ <span class="fragment">$\approx 6$</span></p>
-
-				<p class="fragment">$Sum\left(\frac{k}{N} Samples\right) \cdot \frac{N}{k} \approx Sum(*)$</p>
-
-				<p class="fragment" style="font-weight: bold; margin-top: 60px;">Sampling lets you approximate aggregate values with orders of magnitude less data.</p>
-			</section>
-
-			<section>
-				<h2>Typical OLA Challenges</h2>
-				<dl>
-					<dt>Birthday Paradox</dt>
-						<dd>$Sample(R) \bowtie Sample(S)$ is likely to be empty.</dd>
-					<dt>Stratified Sampling</dt>
-						<dd>It doesn't matter how important they are to the aggregate, rare samples are still rare.</dd>
-					<dt>Replacement</dt>
-						<dd> Does the sampling algorithm converge exactly or asymptotically?</dd>
-				</dl>
-			</section>
-
-			<section>
-				<h2>Replacement</h2>
-				<dl>
-					<dt>Sampling Without Replacement</dt>
-						<dd>... eventually converges to a precise answer.</dd>
-					<dt>Sampling With Replacement</dt>
-						<dd>... doesn't need to track what's been sampled.</dd>
-						<dd>... produces a better behaved estimate distribution.</dd>
-				</dl>
-			</section>
-		</section>
-
 		<section>
 			<section>
 				<img src="graphics/mimir_logo_final.png" />
@@ -214,11 +178,259 @@
 			</section>
 
 			<section>
-				<h2>Key Idea: OLA</h2>
+				<h2>Idea: Use OLA</h2>
+			</section>
+			<section>
+				<h2>Online Aggregation (OLA)</h2>
+
+				<p style="margin-top: 60px;">$Avg(3,6,10,9,1,3,9,7,9,4,7,9,2,1,2,4,10,8,9,7) = 6$</p>
+				<p class="fragment">$Avg(3,6,10,9,1) = 5.8$ <span class="fragment">$\approx 6$</span></p>
+
+				<p class="fragment">$Sum\left(\frac{k}{N} Samples\right) \cdot \frac{N}{k} \approx Sum(*)$</p>
+
+				<p class="fragment" style="font-weight: bold; margin-top: 60px;">Sampling lets you approximate aggregate values with orders of magnitude less data.</p>
+			</section>
+
+			<section>
+				<h2>Typical OLA Challenges</h2>
+				<dl>
+					<dt>Birthday Paradox</dt>
+						<dd>$Sample(R) \bowtie Sample(S)$ is likely to be empty.</dd>
+					<dt>Stratified Sampling</dt>
+						<dd>It doesn't matter how important they are to the aggregate, rare samples are still rare.</dd>
+					<dt>Replacement</dt>
+						<dd> Does the sampling algorithm converge exactly or asymptotically?</dd>
+				</dl>
+			</section>
+
+			<section>
+				<h2>Replacement</h2>
+				<dl>
+					<dt>Sampling Without Replacement</dt>
+						<dd>... eventually converges to a precise answer.</dd>
+					<dt>Sampling With Replacement</dt>
+						<dd>... doesn't need to track what's been sampled.</dd>
+						<dd>... produces a better behaved estimate distribution.</dd>
+				</dl>
+			</section>
+
+			<section>
+				<h2>OLA over GMs</h2>
+				<dl>
+					<dt>Tables are Small</dt>
+						<dd>Compute, not IO is the bottleneck.</dd>
+					<dt>Tables are Dense</dt>
+						<dd>Birthday Paradox and Stratified Sampling irrelevant.</dd>
+					<dt>Queries have High Tree-Width</dt>
+						<dd>Intermediate tables are large.</dd>
+				</dl>
+				<p class="fragment" style="font-weight: bold;">Classical OLA techniques aren't entirely appropriate.</p>
+			</section>
+		</section>
+
+		<section>
+			<section>
+				<h2>(Naive) OLA: Cyclic Sampling</h2>
+			</section>
+
+			<section>
+				<h2>A Few Quick Insights</h2>
+				<ol>
+					<li class="fragment">Small Tables make random access to data possible.</li>
+					<li class="fragment">Dense Tables mean we can sample directly from join outputs.</li>
+					<li class="fragment">Cyclic PRNGs like Linear Congruential Generators can be used to generate a <u>randomly ordered</u>, but <u>non-repeating</u> sequence of integers from $0$ to any $N$ in constant memory.</li>
+				</ol>
+			</section>
+
+			<section>
+				<h2>Linear Congruential Generators</h2>
+				<p>If you pick $a$, $b$, and $N$ correctly, then the sequence:<p>
+				<p>$K_i = (a\cdot K_{i−1}+b)\;mod\;N$</p>
+				<p>will produce $N$ distinct, pseudorandom integers $K_i \in [0, N)$</p>
+			</section>
+
+			<section>
+				<h2>Cyclic Sampling</h2>
+				<p>To marginalize $p(\{X_i\})$...</p>
+				<ol>
+					<li>Init an LCG with a cycle of $N = \prod_i |dom(X_i)|$</li>
+					<li>Use the LCG to sample $\{x_i\} \in \{X_i\}$</li>
+					<li>Incorporate $p(x_i = X_i)$ into the OLA estimate</li>
+					<li>Repeat from 2 until done</li>
+				</ol>
+			</section>
+
+			<section>
+				<h2>Cyclic Sampling</h2>
+				<dl>
+					<dt>Advantages</dt>
+						<dd>Progressively better estimates over time.</dd>
+						<dd>Converges in bounded time.</dd>
+					<dt>Disadvantages</dt>
+						<dd>Exponential time in the number of variables.</dd>
+				</dl>
+			</section>
+
+			<section>
+				<h2>Accuracy</h2>
+				<dl>
+					<dt>Sampling with Replacement</dt>
+						<dd>Chernoff Bounds, Hoeffding Bounds give an $\epsilon-\delta$ guarantee on the sum/avg of a sample <u>with replacement</u>.</dd>
+					<dt>Without Replacement?</dt>
+						<dd class="fragment">Serfling et. al. have a variant of Hoeffding Bounds for sampling without replacement.</dd>
+				</dl>
 				<p></p>
 			</section>
 		</section>
 
+		<section>
+			<section>
+				<h2>Better OLA: Leaky Joins</h2>
+				<p class="fragment">Make Cyclic Sampling into a composable operator</p>
+			</section>
+
+			<section>
+				<img src="graphics/JoinGraph.svg" height="400" style="float:left"/>
+				<table style="float:right">
+					<tr><th>$G$</th><th>#</th><th>$\sum p_{\psi_2}$</th></tr>
+					<tr class="fragment" data-fragment-index="2"><td>1</td><td>3</td><td>0.348</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>2</td><td>4</td><td>0.288</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>3</td><td>4</td><td>0.350</td></tr>
+				</table>
+				<table>
+					<tr><th>$D$</th><th>$G$</th><th>#</th><th>$\sum p_{\psi_1}$</th></tr>
+					<tr class="fragment" data-fragment-index="1"><td>0</td><td>1</td><td>1</td><td>0.126</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>1</td><td>1</td><td>2</td><td>0.222</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>0</td><td>2</td><td>2</td><td>0.238</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>1</td><td>2</td><td>2</td><td>0.050</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>0</td><td>3</td><td>2</td><td>0.322</td></tr>
+					<tr class="fragment" data-fragment-index="3"><td>1</td><td>3</td><td>2</td><td>0.028</td></tr>
+				</table>
+			</section>
+
+			<section>
+				<img src="graphics/JoinGraph.svg" height="400" style="float:left"/>
+				<table style="float:right">
+					<tr><th>$G$</th><th>#</th><th>$\sum p_{\psi_2}$</th></tr>
+					<tr><td>1</td><td>3</td><td>0.348</td></tr>
+					<tr><td>2</td><td>4</td><td>0.288</td></tr>
+					<tr><td>3</td><td>4</td><td>0.350</td></tr>
+				</table>
+				<table>
+					<tr><th>$D$</th><th>$G$</th><th>#</th><th>$\sum p_{\psi_1}$</th></tr>
+					<tr><td>0</td><td>1</td><td>2</td><td>0.140</td></tr>
+					<tr><td>1</td><td>1</td><td>2</td><td>0.222</td></tr>
+					<tr><td>0</td><td>2</td><td>2</td><td>0.238</td></tr>
+					<tr><td>1</td><td>2</td><td>2</td><td>0.050</td></tr>
+					<tr><td>0</td><td>3</td><td>2</td><td>0.322</td></tr>
+					<tr><td>1</td><td>3</td><td>2</td><td>0.028</td></tr>
+				</table>
+			</section>
+
+			<section>
+				<img src="graphics/JoinGraph.svg" height="400" style="float:left"/>
+				<table style="float:right">
+					<tr><th>$G$</th><th>#</th><th>$\sum p_{\psi_2}$</th></tr>
+					<tr><td>1</td><td>4</td><td>0.362</td></tr>
+					<tr><td>2</td><td>4</td><td>0.288</td></tr>
+					<tr><td>3</td><td>4</td><td>0.350</td></tr>
+				</table>
+				<table>
+					<tr><th>$D$</th><th>$G$</th><th>#</th><th>$\sum p_{\psi_1}$</th></tr>
+					<tr><td>0</td><td>1</td><td>2</td><td>0.140</td></tr>
+					<tr><td>1</td><td>1</td><td>2</td><td>0.222</td></tr>
+					<tr><td>0</td><td>2</td><td>2</td><td>0.238</td></tr>
+					<tr><td>1</td><td>2</td><td>2</td><td>0.050</td></tr>
+					<tr><td>0</td><td>3</td><td>2</td><td>0.322</td></tr>
+					<tr><td>1</td><td>3</td><td>2</td><td>0.028</td></tr>
+				</table>
+			</section>
+
+			<section>
+				<h2>Leaky Joins</h2>
+				<ol>
+					<li>Build a normal join/aggregate graph as in variable elimination: One Cyclic Sampler for each Join+Aggregate.</li>
+					<li>Keep advancing Cyclic Samplers in parallel, resetting their output after every cycle so samples "leak" through.</li>
+					<li>When the sampler completes one full cycle with a complete input, mark it complete and stop sampling it.</li>
+					<li>Continue until a desired accuracy is reached or all tables marked complete.</li>
+				</ol>
+			</section>
+
+			<section>
+				<p>There's a bit of extra math to compute $\epsilon-\delta$ bounds by adapting Serfling's results.  It's in the paper.</p>
+			</section>
+		</section>
+
+		<section>
+			<section>
+				<h2>Experiments</h2>
+				<dl>
+					<dt>Microbenchmarks</dt>
+						<dd>Fix time, vary domain size, measure accuracy</dd>
+						<dd>Fix domain size, vary time, measure accuracy</dd>
+						<dd>Vary domain size, measure time to completion</dd>
+					<dt>Macrobenchmarks</dt>
+						<dd>4 graphs from the bnlearn Repository</dd>
+				</dl>
+			</section>
+
+			<section>
+				<h2>Microbenchmarks</h2>
+				<img src="graphics/extended_student.png" />
+				<p><b>Student</b>: A common benchmark graph.</p>
+			</section>
+
+			<section>
+				<h2>Accuracy vs Domain</h2>
+				<img src="graphics/fixed_time.png" height="400"/>
+				<p class="fragment" style="font-weight: bold">VE is binary: It completes, or it doesn't.</p>
+			</section>
+
+			<section>
+				<h2>Accuracy vs Time</h2>
+				<img src="graphics/student_avg.png" height="400"/>
+				<p class="fragment" style="font-weight: bold">CS gets early results faster, but is overtaken by LJ.</p>
+			</section>
+
+			<section>
+				<h2>Domain vs Time to 100%</h2>
+				<img src="graphics/student_scaling.png" height="400"/>
+				<p class="fragment" style="font-weight: bold">LJ is only 3-5x slower than VE.</p>
+			</section>
+
+			<section>
+				<h2>"Child"</h2>
+				<div>
+					<img src="graphics/child.png" style="float:left" height="280">
+					<img src="graphics/child_avg.png" height="350">
+				</div>
+				<p class="fragment" style="font-weight: bold; float:clear">LJ converges to an exact result before Gibbs gets an approx.</p>
+			</section>
+
+			<section>
+				<h2>"Insurance"</h2>
+				<div>
+					<img src="graphics/insurance.png" style="float:left" height="280">
+					<img src="graphics/insurance_avg.png" height="350">
+				</div>
+				<p class="fragment" style="font-weight: bold; float:clear">On some graphs Gibbs is better, but only marginally.</p>
+			</section>
+
+			<section>
+				<h2>More graphs in the paper.</h2>
+			</section>
+		</section>
+
+		<section>
+			<h2>Leaky Joins</h2>
+			<ul>
+				<li>Classical OLA isn't appropriate for GMs.</li>
+				<li><b>Idea 1</b>: LCGs can sample <u>without</u> replacement.</li>
+				<li><b>Idea 2</b>: "Leak" samples through a normal join graph.</li>
+				<li>Compared to both Variable Elim. and Gibbs Sampling, Leaky Joins are often better and never drastically worse.</li>
+			</ul>
+		</section>
+
 		</div></div>
 
 		<script src="../reveal.js-3.1.0/lib/js/head.min.js"></script>