RA to poly translation; corrections 062320
This commit is contained in:
parent
33fc3d74d6
commit
bad5590f2f
|
|
@ -14,6 +14,9 @@
|
|||
\newcommand{\project}{\pi}
|
||||
\newcommand{\union}{\cup}
|
||||
|
||||
%PDBs
|
||||
\newcommand{\ti}{TIDB}
|
||||
|
||||
%Polynomial Reformulation
|
||||
\newcommand{\wbit}{w}
|
||||
\newcommand{\expct}{\mathop{\mathbb{E}}}
|
||||
|
|
|
|||
|
|
@ -1,52 +1,31 @@
|
|||
%root: main.tex
|
||||
\section{Polynomial Formulation}
|
||||
|
||||
Let $\vect_1,\ldots, \vect_\numTup$ be vectors annotating $\numTup$ tuples in a TIDB such that \begin{equation}\vect_i[(\wbit_1,\ldots,\wbit_\numTup)] =
|
||||
\begin{cases} 1 &\wbit_i = 1,\\
|
||||
0 &otherwise.\label{eq:vec-def}
|
||||
\end{cases}
|
||||
\end{equation}
|
||||
Here we define vector indexing by the $\numTup$-bit binary tuple $\wVec = (\wbit_1,\ldots,\wbit_\numTup)$ such that the possible world $\wVec$ is identified by its bit vector binary value.
|
||||
\AR{I do not see why we need to define $\vect$-- everything can be defined without bringing the $\vect$ in. I would recommend that in Section 1 you define things without going into $\vect$-- i.e. state the DB queries and TIDBs and directly define the query polynomial.}
|
||||
|
||||
|
||||
%---We have chosen to ignore the vector formulation
|
||||
%Futher we define the polynomial $\poly(\vect_1,\ldots,\vect_\numTup)$ as an arbitrary polynomial defined over the input vectors, whose addition and multiplication operations are defined as the traditional point-wise vector addition and multiplication. We overload notation and denote the $i^{th}$ world of $\poly$ as $\poly[\wVec]$, where $\poly$ can be viewed as the output annotation vector, and the L-1 norm can be represented as
|
||||
%
|
||||
%\[\norm{\poly}_1 = \sum\limits_{\wVec \in \wSet} \poly[\wVec].\]<---technically incorrect when we consider negative values in \poly
|
||||
%------
|
||||
|
||||
Define $\poly(X_1,\ldots, X_\numTup)$ as a polynomial whose variables represent the tuple annotations of an arbitrary query. While recent research has benefited in viewing the possible worlds problem as a vector $\vect$, notice that \cref{eq:vec-def} is equivalent to $\vect_i[\wVec] = \wbit_i$. With this observation we can further reformulate this problem by viewing $\poly$ as a polynomial over bit values rather than one over vectors, where each element of the input is a bit element of the $\wVec$ bit vector, and we can thus replace each variable of $\poly$ with its corresponding input bit, and solve for the particular world $\wVec$. The output we desire is
|
||||
|
||||
\[\expct_{\wVec}\pbox{\poly(\wVec)} = \sum\limits_{\wVec \in \{0, 1\}^\numTup} \poly(\wVec)\prod_{\substack{i \in [\numTup]\\ s.t. \wElem_i = 1}}\prob_i \prod_{\substack{i \in [\numTup]\\s.t. w_i = 0}}\left(1 - \prob_i\right).\]
|
||||
|
||||
\AR{The above should go into Section 1 (without using $\vect$ of course). And as I mentioned in my comment in Sec 1, you need to figure out a notation for the queries. Check with Oliver on what is standard notation in the PDB literature (unless you know the standard notation in which case eno need to ask Oliver :-).
|
||||
|
||||
Also it might be worthwhile to define a notation for the probability that the world is the specific $\wVec$-- then you can define the expectation for PDB models other than TIDBs.}
|
||||
|
||||
Further, define $\rpoly(X_1,\ldots, X_\numTup)$ as the reduced version of $\poly(X_1,\ldots, X_\numTup)$, of the form
|
||||
\[\rpoly(\wbit_1,\ldots, \wbit_\numTup) = \poly(\wbit_1,\ldots, \wbit_\numTup) \mod \wbit_1^2-\wbit\cdots\mod \wbit_\numTup^2 - \wbit_\numTup.\]
|
||||
\AR{the $w_i$'s should be $X_i$'s. A general comment: to make things clearer, always use $X_i$'s to denote the variabls and $w_i$'s to denote the values that we substitute the variables with.}
|
||||
Intuitively, $\rpoly(\wVec)$ is the expanded sum of products form of $\poly(\wVec)$ such that if any $\wbit_j$ term has an exponent $e > 1$, it is reduced to $1$, i.e. $\wbit_j^e\mapsto \wbit_j$ for any $e > 1$. The usefulness of this reduction will be seen shortly.
|
||||
\AR{The intuition above should be given for the variable setting: i.e. using $X_i$ instead of $w_i$.}
|
||||
\[\rpoly(X_1,\ldots, X_\numTup) = \poly(X_1,\ldots, X_\numTup) \mod \wbit_1^2-\wbit\cdots\mod \wbit_\numTup^2 - \wbit_\numTup.\]
|
||||
|
||||
\AR{You should first state a lemma that show what $\rpoly$ looks like given $\poly(X_1,\ldots, X_\numTup) = \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}}\cdot \prod_{i = 1\text{ s.t. }d_i \geq 1}^\numTup X_i^{d_i}$.}
|
||||
Intuitively, $\rpoly(\textbf{X})$ is the expanded sum of products form of $\poly(\textbf{X})$ such that if any $X_j$ term has an exponent $e > 1$, it is reduced to $1$, i.e. $X_j^e\mapsto X_j$ for any $e > 1$. The usefulness of this reduction will be seen shortly.
|
||||
|
||||
\begin{Lemma}\label{lem:pre-poly-rpoly}
|
||||
When $\poly(X_1,\ldots, X_\numTup) = \sum\limits_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}} \cdot \prod\limits_{\substack{i = 1\\s.t. d_i\geq 1}}^{\numTup}X_i^{d_i}$, we haven then that $\rpoly(X_1,\ldots, X_\numTup) = \sum\limits_{\vct{d} \in \{0,\ldots, D\}^\numTup} q_{\vct{d}}\cdot\prod\limits_{\substack{i = 1\\s.t. d_i\geq 1}}^{\numTup}X_i$.
|
||||
\end{Lemma}
|
||||
|
||||
\AR{The statement below should be typeset as a proposition.}
|
||||
First, note the following fact:
|
||||
\begin{Proposition}
|
||||
\[\text{For all } (\wbit_1,\ldots, \wbit_\numTup) \in \{0, 1\}^\numTup, \poly(\wbit_1,\ldots, \wbit_\numTup) = \rpoly(\wbit_1,\ldots, \wbit_\numTup).\]
|
||||
\end{Proposition}
|
||||
|
||||
\begin{proof}
|
||||
For all $b \in \{0, 1\}$ and all $e \geq 1$, $b^e = 1$.\qed
|
||||
\end{proof}
|
||||
|
||||
\AR{The statement below should be a lemma.}
|
||||
\begin{Property}\label{prop:l1-rpoly-numTup}
|
||||
|
||||
\begin{Lemma}\label{lem:exp-poly-rpoly}
|
||||
The expectation of a possible world in $\poly$ is equal to $\rpoly(\prob_1,\ldots, \prob_\numTup)$.
|
||||
\begin{equation*}
|
||||
\expct_{\wVec}\pbox{\poly(\wVec)} = \rpoly(\prob_1,\ldots, \prob_\numTup).
|
||||
\end{equation*}
|
||||
\end{Property}
|
||||
\end{Lemma}
|
||||
|
||||
\begin{proof}
|
||||
%Using the fact above, we need to compute \[\sum_{(\wbit_1,\ldots, \wbit_\numTup) \in \{0, 1\}}\rpoly(\wbit_1,\ldots, \wbit_\numTup)\]. We therefore argue that
|
||||
|
|
@ -54,54 +33,27 @@ The expectation of a possible world in $\poly$ is equal to $\rpoly(\prob_1,\ldot
|
|||
|
||||
Let $\poly$ be the generalized polynomial, i.e., the polynomial of $\numTup$ variables with highest degree $= D$: %, in which every possible monomial permutation appears,
|
||||
\[\poly(X_1,\ldots, X_\numTup) = \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{d_i}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup X_i^{d_i}\].
|
||||
\AR{It should be $q_{\vct{d}}$ and not $q_{d_i}$ and this change needs to be a propagated.}
|
||||
|
||||
|
||||
Then for expectation we have
|
||||
\begin{align*}
|
||||
\expct_{\wVec}\pbox{\poly(\wVec)} &= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{d_i}\cdot \expct_{\wVec}\pbox{\prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup w_i^{d_i}}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{d_i}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \expct_{\wVec}\pbox{w_i^{d_i}}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{d_i}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \expct_{\wVec}\pbox{w_i}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{d_i}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \prob_i\\
|
||||
&= \rpoly(\prob_1,\ldots, \prob_\numTup)
|
||||
\end{align*}
|
||||
\AR{General comment on when you have a sequence of equalities/inequalities. Always number them and when you are trying to justify them refer to tge specific number. Also I think the 2nd and 3rrd justification in the para below probably need to be switched?}
|
||||
First, by linearity of expectation, the expecation can be pushed all the way inside of the product. Second, note that $w_i \in \{0, 1\}$ which further implies that for any exponent $e \geq 1$, $w_i^e = w_i$. Next, by definition of TIDB, the expectation of a tuple across all possible worlds is indeed its probability. Finally, observe by construction, that $\rpoly(\prob_1,\ldots, \prob_\numTup)$ is exactly the product of probabilities of each variable in each monomial across the entire sum.\AR{The last claim needs more argument-- but this should be easy once you put in the lemma for $\rpoly$ that I asked you to put in one of the comments above.}
|
||||
\begin{align}
|
||||
\expct_{\wVec}\pbox{\poly(\wVec)} &= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}}\cdot \expct_{\wVec}\pbox{\prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup w_i^{d_i}}\label{p1-s1}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \expct_{\wVec}\pbox{w_i^{d_i}}\label{p1-s2}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \expct_{\wVec}\pbox{w_i}\label{p1-s3}\\
|
||||
&= \sum_{\vct{d} \in \{0,\ldots, D\}^\numTup}q_{\vct{d}}\cdot \prod_{\substack{i = 1\\s.t. d_i \geq 1}}^\numTup \prob_i\label{p1-s4}\\
|
||||
&= \rpoly(\prob_1,\ldots, \prob_\numTup)\label{p1-s5}
|
||||
\end{align}
|
||||
|
||||
In steps \cref{p1-s1} and \cref{p1-s2}, by linearity of expectation, the expecation can be pushed all the way inside of the product. In \cref{p1-s3}, note that $w_i \in \{0, 1\}$ which further implies that for any exponent $e \geq 1$, $w_i^e = w_i$. Next, by definition of TIDB, in \cref{p1-s4} the expectation of a tuple across all possible worlds is indeed its probability. Finally, observe \cref{p1-s5} by construction in \cref{lem:pre-poly-rpoly}, that $\rpoly(\prob_1,\ldots, \prob_\numTup)$ is exactly the product of probabilities of each variable in each monomial across the entire sum.
|
||||
|
||||
%Note that for any single monomial, this is indeed the case since the variables in a single monomial are independent and their joint probability equals the product of the probabilities of each variable in the monomial, i.e., for monomial $M$, $\prob[M] = \prod_{x_i \in M}\prob[x_i].$ This is equivalent to the sum of all probabilities of worlds where each variable in $M$ is a $1$. Since $1$ is the identity element, it is also the case that $\prod_{x_i \in M}\prob[x_i] = \ex{M}$. (Note all other terms in the expectation will not contribute since $M$ will equal $0$, and a product containing a factor of $0$ always equals $0$.) It follows then that $\ex{M} = \rpoly(\prob_1,\ldots, \prob_\numTup)$.
|
||||
%
|
||||
%The final result follows by the fact that $\rpoly$ is a sum of monomials, and we can, by linearity of expectation, equivalently push the expectation through the sum and into the monomials.\qed
|
||||
\qed
|
||||
\end{proof}
|
||||
|
||||
%\begin{Property}\label{prop:exp-rpoly}
|
||||
%For the case of general $\prob$, where each tuple in the TIDB is present with probability $\prob$, the expectation of polynomial $\poly$ is equal to $\rpoly(\prob,\ldots, \prob).$
|
||||
%\end{Property}
|
||||
%
|
||||
%\begin{proof}
|
||||
%Note that $\poly$ has an equivalent sum of products form such that $\poly$ is the sum of monomials. By linearity of expectation, the expectation of $\poly$ is equivalent to expectation of each monomial in $\poly$.
|
||||
%
|
||||
%Note further, that for the general monomial, the only operation is product. In the binary (TIDB) setting, if any of the variables in the monomial are zero, then the whole monomial becomes zero. Note that this case contributes nothing to the expectation. Notice also, that if the variables are all one, then their product is one, and the product of the identity element with the product of probabilities is the product of probabilities. This is the only condition which contributes to the expectation. It is therefore necessary to know which worlds the variables in the monomial are all equal to one, and the expectation then is the sum of the probabilities of each world for which the monomial's variables are equal to one. This sum of probabilities is also known as the marginal probability, and this sum is always equal the overall probability of the variables in the monomial. It then stands that the general monomial's expectation is equal to the product of its variable's overall probabilities. The sum of all such expecations is exactly the definition of $\rpoly(\prob,\ldots, \prob)$. Let $M$ be a monomial, $t$ be the number of monomials in $\poly$, and $x_1(\cdots x_v)$ represent the product variable(s) in $M_i$. Then
|
||||
%\begin{equation*}
|
||||
%\ex{M_1 + \cdots + M_t} = \ex{M_1} +\cdots+\ex{M_t}
|
||||
%\end{equation*}
|
||||
%For any $M_i$,
|
||||
%\begin{align*}
|
||||
%&\ex{M_i} = \ex{x_1(\cdots x_v)} = \sum_{(\wElem_1,\ldots, \wElem_N) \in \{0, 1\}^\numTup} x_1(\cdots x_v) \cdot p_1\cdots p_v\\
|
||||
%&=\sum_{\substack{(\wElem_1,\ldots, \wElem_\numTup) \in \{0, 1\}\\\
|
||||
% s.t. \forall i' \in \{j | x_J \in M_i\}\\
|
||||
% \wElem_{i'} = 1}} x_1(\cdots x_v) \prod_{i' \in \{j | x_j \in M_i\}} p_i \prod_{\substack{i'' \not\in \{j | x_j \in M_i\}\\ \wElem_{i''} = 1}}p_{i''}\prod_{\substack{i'' \not\in \{j | x_j \in M_i\}\\ \wElem_{i'''} = 0}} 1 - p_{i'''}\\
|
||||
%&=\prod_{i' \in \{j | x_j \in M_i\}}\prob_{i'}\\
|
||||
%&\implies \ex{M_1} +\cdots+\ex{M_t} = \prod_{i_1 \in \{j | x_j \in M_1\}}\prob_{i_1} +\cdots+\prod_{i_v \in \{j | x_j \in M_v\}} \prob_{i_v}\\
|
||||
%&=\rpoly(\prob_1,\ldots, \prob_\numTup).\qed
|
||||
%\end{align*}
|
||||
%\AH{I just realized, that I could have saved a lot of time by noting that for the case of TIDB, all monomial variables in $M_i$ are independent, and then using linearity of expectation to conclude the proof.}
|
||||
%\end{proof}
|
||||
|
||||
\begin{Corollary}
|
||||
If $\poly$ is given to us in a sum of monomials form, the expectation of $\poly$ ($\ex{\poly}$) can be computed in $O(|\poly|)$, where $|\poly|$ denotes the total number of multiplication/addition operators.
|
||||
\end{Corollary}
|
||||
|
||||
The corollary follows immediately by \cref{prop:l1-rpoly-numTup}.\AR{It does not follow from the statement of \cref{prop:l1-rpoly-numTup} but rather its proof. So this atatement needs its proof as well.}
|
||||
The corollary follows immediately by \cref{lem:exp-poly-rpoly}.\AR{It does not follow from the statement of \cref{lem:exp-poly-rpoly} but rather its proof. So this atatement needs its proof as well.}
|
||||
|
||||
\subsection{When $\poly$ is not in sum of monomials form}
|
||||
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
3) How queries translate into polynomials
|
||||
}
|
||||
|
||||
Given tables $\rel, \reli$, an arbitrary query $\query(\rel)$ over the positive relational operators (SPJU), abusing notation slightly denote the query polynomial as $\poly(X_1,\ldots, X_\numTup)$. The annotation for arbitrary tuple $\tup$ can be viewed as an element of the image of $\rel$, where relation $\rel$ can be thought of as a function with preimage of all tuples in $\rel$, such that $\rel(\tup) = \poly(X_1,\ldots, X_\numTup)$. Further, it is known that the algebraic semiring structure aptly models the translation and computation of query operations into tuple annotation, aka polynomials.
|
||||
Given tables $\rel, \reli$, an arbitrary query $\query(\rel)$ over the positive relational operators (SPJU), abusing notation slightly denote the query polynomial as $\poly(X_1,\ldots, X_\numTup)$. To be clear, $\poly(X_1,\ldots, X_\numTup)$ is a polynomial whose variables represent the tuple annotations of an arbitrary query.The annotation for arbitrary tuple $\tup$ can be viewed as an element of the image of $\rel$, where relation $\rel$ can be thought of as a function with preimage of all tuples in $\rel$, such that $\rel(\tup) = \poly(X_1,\ldots, X_\numTup)$. Further, it is known that the algebraic semiring structure aptly models the translation and computation of query operations into tuple annotation, aka polynomials.
|
||||
To make things more concrete, consider the $\{\mathbb{N}, \times, +, 1, 0\}$ bag semiring. Here the set in which the tuple annotations (computed polynomials) exist is the natural numbers. Query operations are translated into one of the two semiring operators, with $\project$ and $\union$ of agreeing tuples being the equivalent of the '+' opertator in polynomial $\poly$, $\join$ translating into the $\times$ operator, and finally, $\select$ is better modeled as a function that returns either $\rel(\tup)$ or $0$ based on some predicate.
|
||||
|
||||
Consider the translation of relational operators to polynomial operators in greater detail.
|
||||
|
|
@ -25,5 +25,15 @@ Consider the translation of relational operators to polynomial operators in grea
|
|||
\end{cases}
|
||||
\end{align*}
|
||||
|
||||
Considering probabilistic databases, let $\prob(\wVec)$ denote the probability that a given world occurs.
|
||||
The output we desire is over the tuple annotations, i.e. polynomial $\poly(X_1,\ldots, X_\numTup)$ is simply the expectation, i.e.
|
||||
\[\expct_{\wVec}\pbox{\poly(\wVec)} = \sum\limits_{\wVec \in \{0, 1\}^\numTup} \poly(\wVec)\cdot \prob(\wVec).\]
|
||||
|
||||
A specific probabilistic data model is the Tuple Independent Database (\ti). This is a database model in which each table is a set of tuples, each of which are independent of one another, and individually occur with a specific probability, $\prob_\tup$.
|
||||
|
||||
There are features of $\ti$ that we can exploit. Note that a $\ti$ naturally has $2^\numTup$ possible worlds, each of which can be conveniently modeled by an $\numTup$ bit string. The bit-string world value can be used as an index to determine which tuples are present in the $\wVec$ world. We can then write and equivalent expectation for $\ti$ models,
|
||||
|
||||
\[\expct_{\wVec}\pbox{\poly(\wVec)} = \sum\limits_{\wVec \in \{0, 1\}^\numTup} \poly(\wVec)\prod_{\substack{i \in [\numTup]\\ s.t. \wElem_i = 1}}\prob_i \prod_{\substack{i \in [\numTup]\\s.t. w_i = 0}}\left(1 - \prob_i\right).\]
|
||||
|
||||
|
||||
|
||||
\AR{This section needs to be written up {\bf before} you go forwad in the next section. In particular, this section should setup definitions/notation (e.g. how you to denote a general query on TIDB) that will be used later on so better get thet setup first.}
|
||||
|
|
|
|||
Loading…
Reference in a new issue