circuits

approx_alg.tex

@@ -66,9 +66,9 @@ $\expandtree{\etree}$ is the (pure) sum of products expansion of $\etree$, which
 %		&\etree.\type = \tnum \mapsto&& \elist{(\emptyset, \etree.\val)}\\
 %		&\etree.\type = \var \mapsto&& \elist{(\etree.\val, 1)}
 %	\end{align*}
-{\small 
+{\small
 \begin{multline*}
-\expandtree{\etree} = 
+\expandtree{\etree} =
 \begin{cases}
 					\expandtree{\etree_\lchild} \circ \expandtree{\etree_\rchild}		&\textbf{ if }\etree.\type = +\\
 					\left\{(\monom_\lchild \cup \monom_\rchild, \coef_\lchild \cdot \coef_\rchild) ~|~\right.&\\ 							\quad \left.(\monom_\lchild, \coef_\lchild) \in \expandtree{\etree_\lchild}, (\monom_\rchild, \coef_\rchild) \in \expandtree{\etree_\rchild}\right\} 		&\textbf{ if }\etree.\type = \times\\
@@ -179,7 +179,7 @@ In particular, if $\prob_0>0$ and $\gamma<1$ are absolute constants then the abo
 The proof for~\Cref{cor:approx-algo-const-p} can be seen in~\Cref{sec:proofs-approx-alg}.
 The restriction on $\gamma$ is satisfied by \ti (where $\gamma=0$) as well as for all queries of the PDBench \bi benchmark (see \Cref{app:subsec:experiment}).
 Note that (i) tuple presence is independent across blocks, so the corresponding probabilities (and hence $\prob_0$) are independent of the number of blocks, and (ii) \bis model uncertain attributes, so block size (and hence $\gamma$) is a function of the ``messiness'' of a dataset, rather than its size.
-Thus, we expect the corrolary to hold in general.
+Thus, we expect the corollary to hold in general.
 % \AH{I am thinking that perhaps the terminology and presentation of~\Cref{app:subsec:experiment} may need word-smithing to clearly illustrate the $\bi$ benchmarks satisfied--although the substance is already written there.}
 % \AR{Yes! E.g. $\gamma$ is not used at all in~\Cref{app:subsec:experiment}}
 % \AR{{\bf Boris/Oliver:} Is there a way to claim that all probabilities in practice are actually constants: i.e. they do not increase with the number of  tuples?}
@@ -196,7 +196,7 @@ The algorithm to prove~\Cref{lem:approx-alg} follows from the following observat
 Given the above, the algorithm is a sampling based algorithm for the above sum: we sample $(v,c)\in \expandtree{\etree}$ with probability proportional\footnote{We could have also uniformly sampled from $\expandtree{\etree}$ but this gives better parameters.}
 %\AH{Regarding the footnote, is there really a difference?  I \emph{suppose} technically, but in this case they are \emph{effectively} the same.  Just wondering.}
 %\AR{Yes, there is! If we used uniform distribution then in our bounds we will have a parameter that depends on the largest $\abs{coef}$, which e.g. could be dependent on $n$. But with the weighted probability distribution, we avoid paying this price. Though I guess perhaps we can say for the kinds of queries we consider thhese coefficients are all constants?}
-to $\abs{c}$ and compute $Y=\indicator{\monom\mod{\mathcal{B}}\not\equiv 0}\cdot \prod_{X_i\in \var\inparen{v}} p_i$. Taking $\numsamp$ samples and computing the average of $Y$ gives us our final estimate. 
+to $\abs{c}$ and compute $Y=\indicator{\monom\mod{\mathcal{B}}\not\equiv 0}\cdot \prod_{X_i\in \var\inparen{v}} p_i$. Taking $\numsamp$ samples and computing the average of $Y$ gives us our final estimate.
 The number of samples is computed by (see \Cref{app:subsec-th-mon-samp}):
 \begin{equation*}
 2\exp{\left(-\frac{\samplesize\error^2}{2}\right)}\leq \conf \implies\samplesize \geq \frac{2\log{\frac{2}{\conf}}}{\error^2}.

circuits-model-runtime.tex

@@ -3,14 +3,14 @@
 \label{sec:gen}
 In this section, we consider several generalizations/corollaries of our results.
 In particular, in~\Cref{sec:circuits} we first consider the case when the compressed  polynomial is represented by a Directed Acyclic Graph (DAG) instead of an expression tree (\Cref{def:express-tree}) and observe that our results carry over.
-Then, in~\Cref{sec:intro}, we formalize our claim  that a linear algorithm for our problem implies that PDB queries can be answered in the same runtime as deterministic queries.
+Then, we formalize our claim from \Cref{sec:intro} that a linear algorithm for our problem implies that PDB queries can be answered in the same runtime as deterministic queries under reasonable assumptions.
 Finally, in~\Cref{sec:momemts}, we observe how our results can be used to estimate moments other than the expectation.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \subsection{Lineage circuits}
 \label{sec:circuits}
 
-In~\Cref{sec:semnx-as-repr}, we switched to thinking of our query results as polynomials and until now, have focused on thinking of our input as a polynomial. In particular, starting with~\Cref{sec:expression-trees} we considered these polynomials to be represented as an expression tree. However, these do not capture many of the compressed polynomial representations that we can get from query processing algorithms on bags, including the recent work on worst-case optimal join algorithms~\cite{ngo-survey,skew}, factorized databases~\cite{factorized-db}, and FAQ~\cite{DBLP:conf/pods/KhamisNR16}. Intuitively, the main reason is that an expression tree does not allow for `sharing' any intermediate results, which is crucial for these algorithms (and other query processing methods as well).
+In~\Cref{sec:semnx-as-repr}, we switched to thinking of our query results as polynomials and until now, have focused on thinking of our input as a polynomial. In particular, starting with~\Cref{sec:expression-trees} we considered these polynomials to be represented as an expression tree. However, these do not capture many of the compressed polynomial representations that we can get from query processing algorithms on bags, including the recent work on worst-case optimal join algorithms~\cite{ngo-survey,skew}, factorized databases~\cite{factorized-db}, and FAQ~\cite{DBLP:conf/pods/KhamisNR16}. Intuitively, the main reason is that an expression tree does not allow for `sharing' of intermediate results, which is crucial for these algorithms (and other query processing methods as well).
 
 In this section, we represent query polynomials via {\em arithmetic circuits}~\cite{arith-complexity}, a standard way to represent polynomials over fields (particularly in the field of algebraic complexity) that we use for polynomials over $\mathbb N$ in the obvious way.
 We present a formal treatment of {\em lineage circuit}s in~\Cref{sec:circuits-formal}, with only a quick overview to in this section.
@@ -24,7 +24,7 @@ In~\Cref{sec:results-circuits} we argue why results from earlier sections also h
 \subsubsection{Extending our results to lineage circuits}
 \label{sec:results-circuits}
 
-We first note that since expression trees are a special case of linear circuits, all of our hardness results in~\Cref{sec:hard} are still valid for the latter.
+We first note that since expression trees are a special case of linear circuits, all of our hardness results for  in~\Cref{sec:hard} are still valid for the latter.
 
 Observe that \textsc{Approx}\textsc{imate}$\rpoly$ (\Cref{alg:mon-sam} in \Cref{sec:algo}) works for lineage circuits as long as the same guarantees on $\onepass$ and $\sampmon$ (\Cref{lem:one-pass} and \Cref{lem:sample} respectively) hold for lineage circuits as well.
 It turns out that this is the case, simply because both algorithms rely on only one property of expression trees: that each node has two children;
@@ -37,7 +37,12 @@ For a more detailed discussion of why~\Cref{lem:approx-alg} holds for a lineage
 \subsubsection{The cost model}
 \label{sec:cost-model}
 Thus, so far our analysis of the runtime of $\onepass$ has been in terms of the size of the compressed lineage polynomial.
-We now show that this model corresponds to the behavior of a deterministic database by proving that for any union of conjunctive queries, we can construct a compressed lineage polynomial with the same complexity as it would take to evaluate the query on a deterministic \emph{bag} database of the same size as the input PDB. We adopt a minimalistic compute-bound model of query evaluation drawn from the worst-case optimal join literature~\cite{skew,ngo-survey}.
+We now show that this model corresponds to the behavior of a deterministic database by proving that for any union of conjunctive queries, we can construct a compressed lineage polynomial for a query $Q$ and \bi $\pxdb$ in runtime that is linear in the
+runtime that a class of deterministic algorithms take to evaluate $Q(D)$ for any world $\db$ of $\pxdb$ as long as
+there exists a constant $c$ that is independent of the number tuple in the largest world of $\pxdb$  such that $\abs{pxdb} \leq c \cdot \abs{\db}$. In practice, this is often the case because typically the blocks of a \bi represent entities where we are uncertain about their properties and in such a scenario often there are only a limited number of alternatives for each block. Note that all TIDBs trivially fulfill this condition for $c = 1$.
+That is for \bis that fulfill this restriction approximating the expectation of results of SPJU queries is only has a constant factor overhead over deterministic query processing (using one of the algorithms for which we prove the claim).
+% with the same complexity as it would take to evaluate the query on a deterministic \emph{bag} database of the same size as the input PDB.
+We adopt a minimalistic compute-bound model of query evaluation drawn from the worst-case optimal join literature~\cite{skew,ngo-survey}.
 \newcommand{\qruntime}[1]{\textbf{cost}(#1)}
 %
 \noindent\resizebox{1\linewidth}{!}{
@@ -67,7 +72,7 @@ It can be verified that worst-case optimal join algorithms~\cite{skew,ngo-survey
 \subsubsection{Lineage circuits for query plans}
 \label{sec:circuits-formal}
 We now formalize lineage circuits and the construction of lineage circuits for SPJU queries.
-As mentioned earlier, we represent lineage polynomials as arithmetic circuits over $\mathbb N$ with $+$, $\times$.
+As mentioned earlier, we represent lineage polynomials as arithmetic circuits over $\mathbb N$-valued variables with $+$, $\times$.
 A circuit for query $Q$ and $\semNX$-PDB $\pxdb$ is a directed acyclic graph $\tuple{V_{Q,\pxdb}, E_{Q,\pxdb}, \phi_{Q,\pxdb}, \ell_{Q,\pxdb}}$ with vertices $V_{Q,\pxdb}$ and directed edges $E_{Q,\pxdb} \subset {V_{Q,\pxdb}}^2$.
 The sink function $\phi_{Q,\pxdb} : \udom^n \rightarrow V_{Q,\pxdb}$ is a partial function that maps the tuples of the $n$-ary relation $Q(\pxdb)$ to vertices.
 We require that $\phi_{Q,\pxdb}$'s range be limited to sink vertices (i.e., vertices with out-degree 0).
@@ -75,7 +80,7 @@ We require that $\phi_{Q,\pxdb}$'s range be limited to sink vertices (i.e., vert
 A function $\ell_{Q,\pxdb} : V_{Q,\pxdb} \rightarrow \{\;+,\times\;\}\cup \mathbb N \cup \vct X$ assigns a label to each node: Source nodes (i.e., vertices with in-degree 0) are labeled with constants or variables (i.e., $\mathbb N \cup \vct X$), while the remaining nodes are labeled with the symbol $+$ or $\times$.
 We require that vertices have an in-degree of at most two.
 %
-For the specifics on how to construct a  lineage circuit to encode the polynomials of all result tuples for a query and $\semNX$-PDB see~\Cref{app:subsec-rep-poly-lin-circ}.
+For the specifics on how to construct a  lineage circuit to encode the polynomials of all result tuples for a query and $\semNX$-PDB see \Cref{app:subsec-rep-poly-lin-circ}. Note that we can construct lineage circuits for \bis in time linear in the time required for deterministic query processing over a possible world of the \bi under the aforementioned assumption that $\abs{\pxdb} \leq c \cdot \abs{\db}$.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \subsubsection{Circuit size vs. runtime}
@@ -84,7 +89,7 @@ For the specifics on how to construct a  lineage circuit to encode the polynomia
 \newcommand{\bagdbof}{\textsc{bag}(\pxdb)}
 
 We now connect the size of a lineage circuit (where the size of a lineage circuit is the number of vertices in the corresponding DAG) %\footnote{since each node has indegree at most two, this also is the same up to constants to counting the number of edges in the DAG.})
- for a given SPJU query $Q$ and $\semNX$-PDB $\pxdb$ to its $\qruntime{Q,\db}$ where $\bagdbof$ is the bag database consisting of all tuples of $\pxdb$ with multiplicity $1$. We do this formally by showing that the size of the lineage circuit is asymptotically no worse than the corresponding runtime of a large class of deterministic query processing algorithms.
+ for a given SPJU query $Q$ and $\semNX$-PDB $\pxdb$ to its $\qruntime{Q,\db}$ where $\db$ is one of the possible worlds of $\pxdb$. We do this formally by showing that the size of the lineage circuit is asymptotically no worse than the corresponding runtime of a large class of deterministic query processing algorithms.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \begin{lemma}
@@ -97,10 +102,10 @@ Given a $\semNX$-PDB $\pxdb$ and query plan $Q$, the runtime of $Q$ over $\bagdb
 We now have all the pieces to argue that using our approximation algorithm,  the expected multiplicities of a SPJU query can be computed in essentially the same runtime as deterministic query processing for the same query:
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 \begin{Corollary}
-  Given an SPJU query $Q$ over a TIDB $\pxdb$, we can compute a $(1\pm\eps)$-approximation of the expectation for each output tuple with probability at least $1-\delta$ in time
+  Given an SPJU query $Q$ over a \ti $\pxdb$ and let $\db_{max}$ denote the world containing all tuples of $\pxdb$, we can compute a $(1\pm\eps)$-approximation of the expectation for each output tuple with probability at least $1-\delta$ in time
 %
   \[
-    O_k\left(\frac 1{\eps^2}\cdot\qruntime{Q,\bagdbof}\cdot \log{\frac{1}{\conf}}\cdot \log(n)\right)
+    O_k\left(\frac 1{\eps^2}\cdot\qruntime{Q,\db_{max}}\cdot \log{\frac{1}{\conf}}\cdot \log(n)\right)
     \].
 \end{Corollary}
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%