More cleaning up Approx Alg.
This commit is contained in:
parent
baa21d7fef
commit
fd881cd147
|
|
@ -35,7 +35,11 @@ Before proceeding, some useful notation.
|
|||
An expression tree $\polytree$ is an ADT logically viewed as an n-ary tree, whose internal nodes are from the set $\{+, \times\}$, with leaf nodes being either numerical coefficients or variables, but not both.
|
||||
\end{Definition}
|
||||
|
||||
Note that $\polytree$ is the expression tree corresponding to a general polynomial $\poly$, and is therefore not necessarily in the standard monomial basis.
|
||||
Note that $\polytree$ is the expression tree corresponding to a general polynomial $\poly$, and is therefore generally \textit{not} in the standard monomial basis.
|
||||
|
||||
\begin{Definition}[Poly]\label{def:poly-func}
|
||||
Denote $poly(\polytree)$ to be the function that takes as input expression tree $\polytree$ and outputs the polynomial of the same factored form as corresponding to expression tree $\polytree$.
|
||||
\end{Definition}
|
||||
|
||||
\begin{Definition}[Expression Tree Set]\label{def:express-tree-set}$\expresstree{\smb}$ is the set of all possible polynomial expression trees whose standard monomial basis is $\smb$.
|
||||
\end{Definition}
|
||||
|
|
@ -49,10 +53,10 @@ $\expandtree$ is the pure SOP expansion of $\polytree$, where non-distinct monom
|
|||
To illustrate \cref{def:expand-tree} with an example, consider when $\polytree$ models the polynoial $(x + 2y)(2x - y)$. The pure expansion then is $2x^2 - xy + 4xy - 2y^2 = \expandtree$.
|
||||
|
||||
\begin{Definition}[Positive T]\label{def:positive-tree}
|
||||
Let $\abstree$ denote the resulting expression tree when each coefficient $c_i$ in $\polytree$ is exchanged with its absolute value $|c_i|$, and then the resulting $\polytree'$ converted to the standard monomial basis.
|
||||
Let $\abstree$ denote the resulting expression tree when each coefficient $c_i$ in $\polytree$ is exchanged with its absolute value $|c_i|$, and then the resulting $\polytree'$ converted to the expression tree that directly models $poly(\polytree')$, with a root node $+$, whose children consist of all the monomials in the standard monomial basis of $poly(\polytree')$.
|
||||
\end{Definition}
|
||||
|
||||
Using the same polynomial from the above example, $\abstree = (x + 2y)(2x + y) = 2x^2 +xy +4xy + 2y^2 = 2x^2 + 5xy + 2y^2$.
|
||||
Using the same polynomial from the above example, $poly(\abstree) = (x + 2y)(2x + y) = 2x^2 +xy +4xy + 2y^2 = 2x^2 + 5xy + 2y^2$.
|
||||
|
||||
\subsection{Approximating $\rpoly$}
|
||||
|
||||
|
|
@ -62,7 +66,7 @@ Algorithm ~\ref{alg:mon-sam} approximates $\rpoly$ by employing some auxiiliary
|
|||
\subsubsection{Psuedo Code}
|
||||
|
||||
\begin{algorithm}[H]
|
||||
\caption{\textsc{MonomialSample($\polytree$)}}
|
||||
\caption{\textsc{Approximate $\rpoly$($\polytree$)}}
|
||||
\label{alg:mon-sam}
|
||||
\begin{algorithmic}[1]
|
||||
\State $acc \gets 0$
|
||||
|
|
@ -81,17 +85,19 @@ For any query polynomial $\poly(\vct{X})$, an approximation of $\rpoly(\prob_1,\
|
|||
\end{Theorem}
|
||||
|
||||
\begin{Lemma}\label{lem:mon-samp}
|
||||
Algorithm \ref{alg:mon-sam} accurately computes $\frac{\log\frac{1}{\conf}}{\error^2}$ samples, outputting an estimate of $\rpoly(\prob,\ldots, \prob)$ within a multiplicative $1 \pm \error$ error with probability $1 - \conf$.
|
||||
Algorithm \ref{alg:mon-sam} computes $\frac{\log\frac{1}{\conf}}{\error^2}$ samples, outputting an estimate of $\rpoly(\prob,\ldots, \prob)$ within a multiplicative $1 \pm \error$ error with probability $1 - \conf$.
|
||||
\end{Lemma}
|
||||
|
||||
Before the proof, a summary of the sample scheme is necessary. For the input $\polytree$, there are two cases. First, $\polytree$ may already be in SOP form, and then it is trivial to sample from each addend with a weighted probability distribution proportional to each term's coefficient.
|
||||
Before the proof, a brief summary of the sample scheme is necessary. Regardless of the $\polytree$, note that when one samples with a weighted distribution corresponding to the coefficients in $poly(\expandtree)$, it is the same as uniformly sampling over all individual terms of the equivalent polynomial whose terms have coefficients in the set $\{-1, 1\}$, i.e. collapsed monomials are decoupled. Following this reasoning, algorithim ~\ref{alg:one-pass} computes such a weighted distribution and algorithm ~\ref{alg:sample} produces samples accordingly. As a result, from here on, we can consider our sampling scheme to be uniform.
|
||||
|
||||
For the second case, each of the $k$ product terms is sampled from individually, where the final output sample is sampled with a probability that is proportional to its coefficient in $\expandtree$. This is performed by \cref{alg:sample} and its correctness will be argued momentarily. Note, however, that to sample from each of the $k$ products in a POS using a weighted distribution over $\expandtree$ is equivalent to sampling uniformally over all monomials.
|
||||
%each of the $k$ product terms is sampled from individually, where the final output sample is sampled with a probability that is proportional to its coefficient in $\expandtree$. Note, that This is performed by \cref{alg:sample} and its correctness will be argued momentarily. For now it suffices to note that the sampling scheme samples from each of the $k$ products in a POS using a weighted distribution equivalent to sampling uniformly over all monomials.
|
||||
|
||||
\begin{proof}[Proof of Lemma \ref{lem:mon-samp}]
|
||||
Consider $\polytree$ in the standard monomial basis and let $c_i$ be the coefficient of the $i^{th}$ monomial and $\distinctvars_i$ be the number of distinct variables appearing in the $i^{th}$ monomial. Note that sampling each term $t$ in $\polytree$ with probability $\frac{|c_i|}{\abstree(1,\ldots, 1)}$ is the equivalent of sampling uniformly over $\expandtree$. Now consider $\rpoly$ and note that $\coeffitem{i}$ is the value of the $i^{th}$ monomial term in $\rpoly(\prob_1,\ldots, \prob_n)$. Let $m$ be the number of terms in $\expandtree$ and $\coeffset$ to be the set $\{c'_1,\ldots, c'_m\}$ where each $c'_i$ is in $\{-1, 1\}$.
|
||||
The first part of the claim in lemma ~\ref{lem:mon-samp} is trivial, as evidenced in the number of iterations in the for loop.
|
||||
|
||||
Consider now a set of $\samplesize$ random variables $\vct{\randvar}$, where $\randvar_i \sim \unidist{\coeffset}$. Recall that we are estimating for $\rpoly(\prob,\ldots, \prob)$. Then for random variable $\randvar_i$, it is the case that $\expct\pbox{\randvar_i} = \sum_{i = 1}^{\setsize}\frac{c'_i \cdot \prob^{\distinctvars}}{\setsize} = \frac{\rpoly(\prob,\ldots, \prob)}{\abstree(1,\ldots, 1)}$. Let $\hoeffest = \frac{1}{\samplesize}\sum_{i = 1}^{\samplesize}\randvar_i$. It is also true that
|
||||
Next, consider $\expandtree$ and let $c_i$ be the coefficient of the $i^{th}$ monomial and $\distinctvars_i$ be the number of distinct variables appearing in the $i^{th}$ monomial. As discussed above, sampling each term $t$ in $\expandtree$ with probability $\frac{|c_i|}{\abstree(1,\ldots, 1)}$ is the equivalent of sampling uniformly over $\expandtree$. Now consider $\rpoly$ and note that $\coeffitem{i}$ is the value of the $i^{th}$ monomial term in $\rpoly(\prob_1,\ldots, \prob_n)$. Let $m$ be the number of terms in $\expandtree$ and $\coeffset$ to be the set $\{c_1,\ldots, c_m\}.$
|
||||
|
||||
Consider now a set of $\samplesize$ random variables $\vct{\randvar}$, where $\randvar_i \sim \unidist{\coeffset}$ Then for random variable $\randvar_i$, it is the case that $\expct\pbox{\randvar_i} = \sum_{i = 1}^{\setsize}\frac{c'_i \cdot \prob^{\distinctvars}}{\sum_{i = 1}^{\setsize}|c_i|} = \frac{\rpoly(\prob,\ldots, \prob)}{\abstree(1,\ldots, 1)}$. Let $\hoeffest = \frac{1}{\samplesize}\sum_{i = 1}^{\samplesize}\randvar_i$. It is also true that
|
||||
|
||||
\[\expct\pbox{\hoeffest} = \expct\pbox{ \frac{1}{\samplesize}\sum_{i = 1}^{\samplesize}\randvar_i} = \frac{1}{\samplesize}\sum_{i = 1}^{\samplesize}\expct\pbox{\randvar_i} = \frac{1}{\samplesize}\sum_{i = 1}^{\samplesize}\frac{1}{\setsize}\sum_{j = 1}^{\setsize}\frac{c'_i \cdot \prob^{\distinctvars}}{\setsize} = \frac{\rpoly(\prob,\ldots, \prob)}{\abstree(1,\ldots, 1)}.\]
|
||||
|
||||
|
|
@ -109,30 +115,37 @@ Solving for the number of samples $\samplesize$ we get
|
|||
|
||||
Equation \cref{eq:hoeff-1} results computing the sum in the denominator of the exponential. Equation \cref{eq:hoeff-2} is the result of dividing both sides by $2$. Equation \cref{eq:hoeff-3} follows from taking the reciprocal of both sides, and noting that such an operation flips the inequality sign. We then derive \cref{eq:hoeff-4} by the taking the base $e$ log of both sides, and \cref{eq:hoeff-5} results from reducing common factors. We arrive at the final result of \cref{eq:hoeff-6} by simply multiplying both sides by the reciprocal of the RHS fraction without the $\samplesize$ factor.
|
||||
|
||||
By Hoeffding, then algorithm ~\ref{alg:mon-sam} takes the correct number of samples, and this concludes the proof of lemma ~\ref{lem:mon-samp}.
|
||||
|
||||
\end{proof}
|
||||
|
||||
|
||||
\subsubsection{Run-time Analysis}
|
||||
|
||||
\subsection{Sampling Algorithm}
|
||||
\subsection{OnePass Algorithm}
|
||||
\subsubsection{Description}
|
||||
Auxiliary algorithm \textit{Sample} performs the brunt of the work in \textit{Monomial Sample}. \textit{Sample} takes $\polytree$ as input and performs the equivalent of outputting a sample $\randvar_i$ such that $\randvar_i \sim Uniform(S)$, where $S$ represents the multiset of monomials in $\expandtree$. While one cannot compute $\expandtree$ in time better than $O(N^k)$, the algorithm uses a technique on $\polytree$ which produces a uniform sample from $\expandtree$ without ever materializing $\expandtree$.
|
||||
Auxiliary algorithm ~\ref{alg:one-pass} has the responsibility of computing the weighted (uniform) distribution over $\expandtree$. This consists of two parts. Computing the sum over the absolute values of each coefficient ($\abstree(1,\ldots, 1)$), and computing the distribution over each monomial term of $\expandtree$, without ever materializing $\expandtree$.
|
||||
|
||||
The input $\polytree$ can be seen as an expression tree, with leaf nodes being the variables of $\polytree$ and inner nodes being the polynomial operations of $+$ or $\times$. For clarity, consider when $\polytree = (x_1 + x_2)(x_1 - x_2) + x_2^2$. The expression tree for $\polytree$ would look like $+\left(\times\left(+\left(x_1, x_2\right), +\left(x_1, -x_2\right)\right), \times\left(y, y\right)\right)$.
|
||||
The input to algorithm ~\ref{alg:one-pass} is $\polytree$. For concreteness, consider the example when $poly(\polytree) = (x_1 + x_2)(x_1 - x_2) + x_2^2$. The expression tree $\polytree$ would then be $+\left(\times\left(+\left(x_1, x_2\right), +\left(x_1, -x_2\right)\right), \times\left(y, y\right)\right)$.
|
||||
|
||||
\AH{A tree diagram would work much better, but to do that it appears that I need to spend time learning the tikz package, which I haven't had time for yet.}
|
||||
|
||||
Algorithm ~\ref{alg:one-pass} performs two important tasks in one pass over expression tree $\polytree$. One of those tasks is to compute $\abstree(1,\ldots, 1)$ in $O(|\poly|)$ time. Note, that $\abstree(1,\ldots, 1)$ is indeed the sum of coefficients in $\abstree$, or equivalently the number of terms in $\expandtree$.
|
||||
To compute $\abstree(1,\ldots, 1)$, algorithm ~\ref{alg:one-pass} makes a bottom-up traversal of $\polytree$ and performs the following computations. For a leaf node whose value is a coefficient, the value is saved. When a $+$ node is visited, the coefficient values of its children are summed. Finally, for the case of a $\times$ node, the coefficient values of the children are multiplied.
|
||||
|
||||
Secondly, algorithm ~\ref{alg:one-pass} recursively visits nodes, where it performs the following operations. Given a leaf node, the algorithm stores the coefficient $c_i$ absolute value, denoted $|c_i|$. When it stops at a $+$ node, it memoizes the probabilities proportional to the partial values of the subtree children of the parent $+$ node. For the case when it encounters an inner $\times$ node, the partial values of both subtree children are multiplied.
|
||||
Algorithm ~\ref{alg:one-pass} computes the weighted (uniform) probability distribution in the same bottom-up traversal. When a leaf node is encountered, its value is saved if it is a coefficient. When a $\times$ node is visited, the coefficient values of its children are multiplied. When a $+$ node is visited, the algorithm computes and saves the relative probabilities of each one of its children. This is done by taking the sum of its children's coefficient absolute values, and for each child, dividing the child's coefficient absolute value by that sum. Note the difference in treatment between $+$ and $\times$ nodes, that the former stores probabilities for its children while the latter stores a probability measure for itself.
|
||||
|
||||
For the running example, after the first pass, \cref{alg:one-pass} would have learned to sample the two children of the root $+$ node with $P\left(\times\left(+\left(x_1, x_2\right), +\left(x_1, -x_2\right)\right)\right) = \frac{4}{5}$ and $P\left(\times\left(x_2, x_2\right)\right) = \frac{1}{5}$. Similarly, the two inner $+$ nodes of the root's left child, call them $+_1$ and $+_2$, using $l$ for left child and $r$ for right child are $P_{+_1}(l) = P_{+_1}(r) = P_{+_2}(l) = P_{+_2}(r) = \frac{1}{2}$. Note that in this example, the sampling probabilities for the children of each inner $+$ node are equal to one another because both parents have the same number of children, and, in each case, the children of each parent $+$ node share the same $|c_i|$.
|
||||
|
||||
Algorithm ~\ref{alg:sample} then uniformly selects a monomial from $\expandtree$ by the following recursive method. For each leaf node, the monomial is returned with a coefficient reduced to either $\{-1, 1\}$ depending on its sign. For a parent $+$ node, a subtree is chosen over the previously computed weighted sampling distribution. When a parent $\times$ node is visited, the monomials are combined into one monomial. The algorithm concludes outputting $sign(c_i)\cdot\prob^{d_i}$.
|
||||
\AH{{\bf Next:} 1) Discuss terms used in algo data structure $\polytree$.
|
||||
2) Add algo comments.
|
||||
3) Move on to algo Sample.
|
||||
4) Add comments.
|
||||
5) Prove correctness for all algos.
|
||||
6) Don't forget to do run-time analysis.}
|
||||
|
||||
\subsubsection{Psuedo Code}
|
||||
|
||||
\begin{algorithm}
|
||||
\begin{algorithm}[h!]
|
||||
\caption{OnePass($\polytree$)}
|
||||
\label{alg:one-pass}
|
||||
\begin{algorithmic}[1]
|
||||
|
|
@ -158,6 +171,14 @@ Algorithm ~\ref{alg:sample} then uniformly selects a monomial from $\expandtree$
|
|||
\end{algorithmic}
|
||||
\end{algorithm}
|
||||
|
||||
\subsection{Sample Algorithm}
|
||||
|
||||
Algorithm ~\ref{alg:sample} takes $\polytree$ as input and performs the equivalent of outputting a sample $\randvar_i$ such that $\randvar_i \sim Uniform(S)$, where $S$ represents the multiset of monomials in $\expandtree$. While one cannot compute $\expandtree$ in time better than $O(N^k)$, the algorithm uses a technique on $\polytree$ which produces a uniform sample from $\expandtree$ without ever materializing $\expandtree$.
|
||||
|
||||
Algorithm ~\ref{alg:sample} then uniformly selects a monomial from $\expandtree$ by the following top-down traversal. For each leaf node, the monomial is returned with a coefficient reduced to either $\{-1, 1\}$ depending on its sign. For a parent $+$ node, a subtree is chosen over the previously computed weighted sampling distribution. When a parent $\times$ node is visited, the monomials are combined into one monomial. The algorithm concludes outputting $sign(c_i)\cdot\prob^{d_i}$.
|
||||
|
||||
\subsubsection{Pseudo Code}
|
||||
|
||||
\begin{algorithm}
|
||||
\caption{Sample($\polytree$)}
|
||||
\label{alg:sample}
|
||||
|
|
|
|||
Loading…
Reference in a new issue