Done with pass on Sec 11

sop.tex

@@ -1,7 +1,7 @@
 %root--main.tex
 \section{Sum of Products Analysis}
 
-\AR{You should do the analysis for $\lambda(j,j')$ instead of just  $\sigma_j^2=\lambda(j,j)$. The former is not that different from what you have below and you'll need to do it when comoputing $\sigma^2$ in any case.}
+\AR{You should do the analysis for $\lambda(j,j')$ instead of just  $\sigma_j^2=\lambda(j,j)$. The former is not that different from what you have below and you'll need to do it when computing $\sigma^2$ in any case.}
 We now seek to bound the variance of a $\prodsize$-way join.
 \begin{align}
 &\sigsq_j = \ex{\est_j \cdot \overline{\est_j}} - \ex{\est_j} \cdot \ex{\overline{\est_j}} \nonumber\\
@@ -22,8 +22,8 @@ Before proceeding, we introduce some notation  and terminology that will aid in
 
 \AR{Sorry I missed this earlier but I do not think you need the $T_2$ term since you ``cancel" out their sum from the corresponding terms in the sum of the $T_1$ terms.}
 
-We will use the vocabulary 'term' to denote $\prod_{i = 1}^{\prodsize}\vect_i(\wElem_i)\vect_i(\wElem_i') \cdot\left(\term_1 - \term_2\right)$ given a specific set of world values.  To say that a term survives \AR{Yoou should not care about whether the $T_1$ term survives or not. See the above comment on why.} the expectation is to mean that $\term_1 - \term_2 \neq 0$.  Note, that the only terms that survive the expectation above are mappings of $w_i = w'_j = w$ for $i, j \in [\prodsize]$, such that each $w_i$ has a match, i.e., no $w_i$ or $w'_j$ stands alone without a matching world in its complimentary set.  In other words, the set of values in $\wElem_1,\ldots,\wElem_k$ has a bijective mapping to the set of values in $\wElem'_1,\ldots,\wElem'_k$.
-\AR{I am not sure this last sentence is needed here or not. I think it probably is more confusing withouot the details that are forthcoming. I think you can just give a foward pointer to  Lemma~\ref{lem:sig-j-survive} here.}
+We will use the vocabulary 'term' to denote $\prod_{i = 1}^{\prodsize}\vect_i(\wElem_i)\vect_i(\wElem_i') \cdot\left(\term_1 - \term_2\right)$ given a specific set of world values.  To say that a term survives \AR{You should not care about whether the $T_1$ term survives or not. See the above comment on why.} the expectation is to mean that $\term_1 - \term_2 \neq 0$.  Note, that the only terms that survive the expectation above are mappings of $w_i = w'_j = w$ for $i, j \in [\prodsize]$, such that each $w_i$ has a match, i.e., no $w_i$ or $w'_j$ stands alone without a matching world in its complimentary set.  In other words, the set of values in $\wElem_1,\ldots,\wElem_k$ has a bijective mapping to the set of values in $\wElem'_1,\ldots,\wElem'_k$.
+\AR{I am not sure this last sentence is needed here or not. I think it probably is more confusing without the details that are forthcoming. I think you can just give a forward pointer to  Lemma~\ref{lem:sig-j-survive} here.}
 
 %\subsection{M-tuples}
 %\begin{Definition}
@@ -36,7 +36,7 @@ We will use the vocabulary 'term' to denote $\prod_{i = 1}^{\prodsize}\vect_i(\w
 
 \subsection{f, f'}
 \begin{Definition}
-Define and then fix the lexographical ordering of distinct world elements to be the total ordering of the elements in $[\dist]$ such that $\forall i < j \in [\dist], \widetilde{\wElem_i} < \widetilde{\wElem_j}$.
+Define and then fix the lexicographical ordering of distinct world elements to be the total ordering of the elements in $[\dist]$ such that $\forall i < j \in [\dist], \widetilde{\wElem_i} < \widetilde{\wElem_j}$.
 %Given a fixed order $\wSet_{\order}: \left(\wSet, \wSet\right)\mapsto \mathbb{B}$ of possible worlds, define the lexographical order of distinct worlds $\wSet_\dist$ to be the ordering which complies to the identity mapping of elements in $[\prodsize]$ to elements in $[\dist]$ up to $\dist$, such that .  In other worlds, $\forall \wElem, \wElem' \in \wSet_\dist, \widetilde{\wElem} < \wElem' \leftrightarrow \wSet_{\order}\left(\wElem, \wElem'\right) = T$.
 \end{Definition}
 To help describe all possible matchings we introduce functions $f$ and $f'$.
@@ -79,7 +79,7 @@ To avoid double counting, we impose an ordering on the set of functions $f, f'$
 For every $i, j \in [\dist]~|~ i < j$, the numerical value of the concatenation of the numerically ordered elements of $f^{-1}(i)$ < the numerical value of the concatenation of the numerically ordered elements of $f^{-1}(j)$, where $<$ is the order of the natural numbers.
 \end{Definition}
 
-\AR{As I mentioned in my email, this definition is too confusing. If you need like 1/4th of a page of explain a defintion you should re-consider if you should use it in the first place. I think putting a lex order on $\tilde{w}_1\prec \tilde{w}_2\prec \cdots \prec \tilde{w}_m$ is a better way to go.}
+\AR{As I mentioned in my email, this definition is too confusing. If you need like 1/4th of a page of explain a definition you should re-consider if you should use it in the first place. I think putting a lex order on $\tilde{w}_1\prec \tilde{w}_2\prec \cdots \prec \tilde{w}_m$ is a better way to go.}
 
 We illustrate with an example.  Consider a join of $k = 3$ tuples, where $\dist = 2$, and we have that $f^{-1}(1) = 1$ and $f^{-1}(2) = 2$.  Imposing the above ordering yields the following set of unique functions:
 \begin{align*}
@@ -97,7 +97,7 @@ f_3 = \begin{cases}
 	\end{cases}
 \end{align*}
 The above mappings satisfy the ordering constraint so that for $f_1$, $1 < 23$, for $f_2$, $2 < 13$, and for $f_3$, $3 < 12$.
-Note that above orderings share no symmetry, while the symmetrical versions of the above, which are the orderings for the case when $f^{-1}(1) = 2$ and $f^{-1}(2) = 1$, break our above ordering requirements, and are therefore disallowed, thus avoiding double counting.  Another way of saying this is that the preimage sizes follow the natural order of their respective counterparts in the image.  For the case when the two are equal, we need a more defined order, and can distinguish using the same ideaology as first described.
+Note that above orderings share no symmetry, while the symmetrical versions of the above, which are the orderings for the case when $f^{-1}(1) = 2$ and $f^{-1}(2) = 1$, break our above ordering requirements, and are therefore disallowed, thus avoiding double counting.  Another way of saying this is that the preimage sizes follow the natural order of their respective counterparts in the image.  For the case when the two are equal, we need a more defined order, and can distinguish using the same idea as first described.
 
 \begin{Lemma}\label{lem:sig-j-survive}
 The only terms surviving $\term_1 - \term_2$ are those with $f, f'$ matching, where $\forall j \in[\dist], \dMap{\wElem_j} = \dMap{\wElem'_j}$.
@@ -123,13 +123,16 @@ In the above, since we have more than pairwise independence for $\wElem \neq \wE
 \AR{Proof environment for Lemma\cref{lem:sig-j-survive} should start here.}
 
 \AR{Um, the proof below is bit of a mess. This needs to be re-written. Below are some suggestions.}
-\AR{First some typos/things that are incorrect below-- note this is \textbf{not} an exhaustive list. (1) In the proof below the $w_i$ and $w'_i$ should be $\tilde{w}_i$ and $\tilde{w'}_i$ repectively. (2) The expression for $T_1$ below is incorrect since it seems to assume that all the pre-image sizes are $1$-- the expression for $T_2$ is fine except the $j_i$ terms are not defined. However, ``taking out" one term for $\tilde{w'}_{m'}$ for $T_2$ is incorrect since e.g. we could have the pre-image of $m'$ have size $>1$. (3) The proof below never explicitly argues why the condition $\dMap{\wElem_j} = \dMap{\wElem'_j}$ is needed.}
-\AR{Here is how I recoommend that you re-write the proof. First as mentioned earlier, you should only consider the $T_1$ terms (as you account for the $T_2$ terms later on. Second you shoould first start off by re-stating the $T_1$ term like so. Consider the ``generic term"--
+\AR{First some typos/things that are incorrect below-- note this is \textbf{not} an exhaustive list. (1) In the proof below the $w_i$ and $w'_i$ should be $\tilde{w}_i$ and $\tilde{w'}_i$ respectively. (2) The expression for $T_1$ below is incorrect since it seems to assume that all the pre-image sizes are $1$-- the expression for $T_2$ is fine except the $j_i$ terms are not defined. However, ``taking out" one term for $\tilde{w'}_{m'}$ for $T_2$ is incorrect since e.g. we could have the pre-image of $m'$ have size $>1$. (3) The proof below never explicitly argues why the condition $\dMap{\wElem_j} = \dMap{\wElem'_j}$ is needed.}
+\AR{Here is how I recommend that you re-write the proof. First as mentioned earlier, you should only consider the $T_1$ terms (as you account for the $T_2$ terms later on. Second you should first start off by re-stating the $T_1$ term like so. Consider the ``generic term"--
 \[T_1(\tilde{w}_{f(1)},\dots, \tilde{w}_{f(m)}, \tilde{w'}_{f'(1)},\dots, \tilde{w'}_{f'(m')}).\]
-Then re-write the what the above term is based on the exact definiion (BTW I'm dropping the $\mathbf{E}$ terms for convenience but they should be all there below.) In particular, the above term by definition is exactly
-\[\prod_{i=1}^k s(\tilde{w}_{f(i)})\cdot s(\tilde{w'}_{f'(i)}).\]
+Then re-write the what the above term is based on the exact definition (BTW I'm dropping the $\mathbf{E}$ terms for convenience but they should be all there below.) In particular, the above term by definition is exactly
+\[\prod_{i=1}^k s(\tilde{w}_{f(i)})\cdot \overline{s(\tilde{w'}_{f'(i)})}.\]
+Now re-write the above in terms of ``powers" of distinct worlds:
+\[ (\prod_{i=1}^m s(\tilde{w}_{i})^{|f^{-1}(i)|})\cdot \overline{(\prod_{j=1}^m s(\tilde{w'}_j)^{|f^{-1}(j)|})}\]
+Now once you have the above expression, then it will be much easier to argue why if any of the matching conditions are not satisfied then the expression is $0$. I also believe that working with the above expression will also make it more ``obvious" as to why the different conditions are required. Currently the arguments below do not explicitly bring this out...
 }
-To prove that \cref{lem:sig-j-survive} is true, consider what the expectation looks like when $f, f'$ are not matching. Looking at the first condition for $f, f'$ to be matching when $\dist \neq \dist'$ note that since $\dist \neq \dist'$ we know that one set of variables has at least one more distinct world than the other set of variables.  Also, to be explicit, $\wElem_1\ldots\wElem_\dist, \wElem_1'\ldots\wElem_{\dist'}'$ are distinct world values sucht that $\forall i \neq j \in [\dist], \wElem_i = \wElem_i' \neq \wElem_j = \wElem_j'$.  To make things easier, assume that $\dist < \dist'$.  The opposite case of $\dist > \dist'$ has a symmetrical proof.  Fixing variables $\wElem_1\ldots\wElem_\dist, \wElem_1'\ldots\wElem_\dist$, in both $\term_1$ and $\term_2$ we have one extra distinct value, $\wElem_{\dist'}'$.  This distinct term cancels out all the other values in the expectations.
+To prove that \cref{lem:sig-j-survive} is true, consider what the expectation looks like when $f, f'$ are not matching. Looking at the first condition for $f, f'$ to be matching when $\dist \neq \dist'$ note that since $\dist \neq \dist'$ we know that one set of variables has at least one more distinct world than the other set of variables.  Also, to be explicit, $\wElem_1\ldots\wElem_\dist, \wElem_1'\ldots\wElem_{\dist'}'$ are distinct world values such that $\forall i \neq j \in [\dist], \wElem_i = \wElem_i' \neq \wElem_j = \wElem_j'$.  To make things easier, assume that $\dist < \dist'$.  The opposite case of $\dist > \dist'$ has a symmetrical proof.  Fixing variables $\wElem_1\ldots\wElem_\dist, \wElem_1'\ldots\wElem_\dist$, in both $\term_1$ and $\term_2$ we have one extra distinct value, $\wElem_{\dist'}'$.  This distinct term cancels out all the other values in the expectations.
 
 \begin{align}
 \term_1 - \term_2 = &\ex{\sine(\wElem_1)\conj{\sine(\wElem_1)}\cdot\ldots\cdot\sine(\wElem_\dist)\conj{\sine(\wElem_\dist)}\cdot\conj{\sine(\wElem'_{\dist'})}} - \ex{\prod_{i = 1}^{\dist}\sine(\wElem_i)^{j_i}}\ex{\left(\prod_{i = 1}^{\dist}\sine(\wElem_i)^{j_i}\right) \cdot \conj{\sine(\wElem'_{\dist'})}}\\
@@ -173,7 +176,7 @@ The above means that we will have at least two world values  that don't match, i
 		\ex{\prod_{i'' = 1 s.t. i'' \neq i}^{\dist}\sine(\wElem_{i''})}\cdot \ex{\sine(\wElem_{i})}\cdot\ex{\prod_{i'' = 1 s.t. i'' \neq i'}^{\dist}\conj{\sine(\wElem_{i''}')}}\cdot \ex{\conj{\sine(\wElem_{i'}')}}\\
 = &0.
 \end{align}
-By the same arguments as before, we have at least one distinct world value in each expecation, which by independence of random variables allows us to push the expectations into the product, and then by \cref{lem:exp-prod-rand-roots} and \cref{lem:exp-sine} produce a zero in each product, yielding a value of zero.\qed
+By the same arguments as before, we have at least one distinct world value in each expectation, which by independence of random variables allows us to push the expectations into the product, and then by \cref{lem:exp-prod-rand-roots} and \cref{lem:exp-sine} produce a zero in each product, yielding a value of zero.\qed
 %To be clear, if the set of preimage sizes do not match, then we have for both $f, f'$ at least two preimage mappings to their respective distinct variables whose corresponding sizes are both unequal.  In the notation above, we set the unmatching cardinalities to $\dupSize_s$, etc. to reference later on.  Note that since $\dist$ is the same for both $f, f'$, the disagreement in cardinalities evens out across variables.  For the sake of argument, define $\dupSize_s > \dupSize_s', \dupSize_t' > \dupSize_t$.
 %
 %Translating this to $\term_1$ and $\term2$ we have
@@ -209,7 +212,8 @@ We now seek to show that when $f, f'$ are matching, that $\term_1 - \term_2$ wil
 
 Using the above definitions, we can now present the variance bounds for $\sigsq_j$ based on \eqref{eq:sig-j-distinct}.
 
-By the fact that the expectations cancel when $\forall i, i', j, j'\in [\prodsize], \wElem_i = \wElem_j = \wElem_{i'}' = \wElem_{j'}' = \wElem$, we can rid ourselves of the case when there exists only one distinct world value.  We then need to sum up all the $\dist$ distinct world value possibilities for $\dist \in [2, \prodsize]$.  Note that the number of distinct values $\dist$ affects the randomness of the hash function $\hfunc$.  E.g. only $\dist = 2$ distinct values will  yield $\frac{1}{\sketchCols} \cdot \frac{1}{\sketchCols} = \frac{1}{\sketchCols^2} = \frac{1}{\sketchCols^\dist}$.  By lemma \ref{lem:sig-j-survive} and equation \eqref{eq:sig-j-distinct} we get
+By the fact that the expectations cancel when $\forall i, i', j, j'\in [\prodsize], \wElem_i = \wElem_j = \wElem_{i'}' = \wElem_{j'}' = \wElem$, we can rid ourselves of the case when there exists only one distinct world value.\AR{This is where you are subtracting off the $T_2$ terms, which is why you do not need to consider them in the arguments above.}  We then need to sum up all the $\dist$ distinct world value possibilities for $\dist \in [2, \prodsize]$.  Note that the number of distinct values $\dist$ affects the randomness of the hash function $\hfunc$.  E.g. only $\dist = 2$ distinct values will  yield $\frac{1}{\sketchCols} \cdot \frac{1}{\sketchCols} = \frac{1}{\sketchCols^2} = \frac{1}{\sketchCols^\dist}$.  By lemma \ref{lem:sig-j-survive} and equation \eqref{eq:sig-j-distinct} we get
+\AR{Um, no this is not an argument for general situation. You need to explicitly argue how each term in Lemma~\eqref{eq:sig-j-distinct} fares under Lemma~\ref{lem:sig-j-survive} and then how those give you the expression below. In particular, you should explicitly argue what happens to the indicator variable terms. Also doing this for $\lambda(j,j')$ would mean you'll have to deal with those explicitly in any case.}
 %
 %\begin{equation*}
 %\frac{1}{\sketchCols^2}\sum_{\dMap{\wElem_1}, \dMap{\wElem_2}}\prod_{i = 1}^{\prodsize}\vect_i(\dMap{\wElem_{f(i)}})\vect_i(\dMap{\wElem_{f'(i)}}).
@@ -225,3 +229,5 @@ By the fact that the expectations cancel when $\forall i, i', j, j'\in [\prodsiz
 \begin{equation*}
 \sigsq_j = \sum_{\dist \in [2, \prodsize]} \frac{1}{B^\dist} \sum_{\dMap{w_1}\ldots\widetilde{w_\dist}\in W} \sum_{\substack{f, f',\\\match{f}{f'}}} \prod_{i = 1}^{\prodsize} v_i(\dMap{w_{f(i)}}) v_i(\dMap{w_{f'(i)}})
 \end{equation*}
+
+\AR{Remaining TODOs: (1) Give expression for general $\sigma^2$, i.e. deal with the general $\lambda(j,j')$ term. (2) Show how to use the analysis for general $k$-product to handle generic  SoP expressions-- the expectation arguments would just follow from the above and linearity of expectation but the variance bounds might need a bit of extra work.}