paper-BagRelationalPDBsAreHard/mult_distinct_p.tex

116 lines
9.0 KiB
TeX
Raw Normal View History

2020-12-04 13:14:12 -05:00
%root:main.tex
%!TEX root=./main.tex
2021-09-15 23:51:29 -04:00
\section{Hardness of Exact Computation}
2020-12-13 21:53:22 -05:00
\label{sec:hard}
2022-06-07 23:26:24 -04:00
In this section, we will prove the hardness results claimed in Table~\ref{tab:lbs} for a specific (family) of hard instances $(\qhard^k,\pdb)$ for \Cref{prob:bag-pdb-poly-expected} where $\pdb$ is a $1$-\abbrTIDB.
2022-06-02 00:11:18 -04:00
Note that this implies hardness for \abbrCTIDB\xplural $\inparen{\bound\geq1}$
%; \Cref{prob:bag-pdb-poly-expected} cannot be done in $\bigO{\qruntime{\optquery{\query},\tupset,\bound}}$ runtime. The results also apply to
as well as \abbrOneBIDB. % and other \abbrPDB\xplural.
2020-12-18 12:23:13 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2022-06-03 22:08:17 -04:00
%\subsection{Preliminaries}\label{sec:hard:sub:pre}
2022-06-07 23:26:24 -04:00
Our hardness results are based on (exactly) counting the number of (not necessarily induced) subgraphs in $G$ isomorphic to $H$. Let $\numocc{G}{H}$ denote this quantity. We think of $H$ as being of constant size and $G$ as growing.
2022-06-03 22:08:17 -04:00
In particular, we will consider computing the following counts (given $G$ in its adjacency list representation): $\numocc{G}{\tri}$ (the number of triangles), $\numocc{G}{\threedis}$ (the number of $3$-matchings), and the latter's generalization $\numocc{G}{\kmatch}$ (the number of $k$-matchings). We use $\kmatchtime$ to denote the optimal runtime of computing $\numocc{G}{\kmatch}$ exactly. Our results in \Cref{sec:multiple-p} are based on the following known (conditional) hardness results:
2020-12-11 19:50:53 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{Theorem}[\cite{k-match}]
2020-12-09 00:00:04 -05:00
\label{thm:k-match-hard}
Given positive integer $k$ and undirected graph $G=(\vset,\edgeSet)$ with no self-loops or parallel edges, $\kmatchtime\ge \littleomega{f(k)\cdot |\edgeSet|^c}$ for any function $f$ and any constant $c$ independent of $\abs{E}$ and $k$ (assuming $\sharpwzero\ne\sharpwone$).
\end{Theorem}
2020-12-11 19:50:53 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2022-05-02 08:10:58 -04:00
%\begin{hypo}\label{conj:known-algo-kmatch}
%There exists an absolute constant $c_0>0$ such that for every $G=(\vset,\edgeSet)$, we have $\kmatchtime \ge \Omega\inparen{|E|^{c_0\cdot k}}$ for large enough $k$.
%\end{hypo}
2022-06-02 09:43:04 -04:00
%<<<<<<< HEAD
%\begin{hypo}[~\cite{10.1109/FOCS.2014.22}]\label{conj:known-algo-kmatch}
%For every $G=\inparen{\vset, \edgeSet}$, $\kmatchtime\ge n^{\Omega\inparen{k/\log{k}}}$.
%\end{hypo}
%=======
\begin{Theorem}[~\cite{10.1109/FOCS.2014.22}]\label{conj:known-algo-kmatch}
2022-06-09 20:27:28 -04:00
Given positive integer $k$ and undirected graph $G=(\vset,\edgeSet)$, $\kmatchtime\ge |\vset|^{\Omega\inparen{k/\log{k}}}$ (assuming ETH).
2022-06-02 00:11:18 -04:00
\end{Theorem}
2022-05-02 08:10:58 -04:00
2022-06-02 00:11:18 -04:00
%We note that the above conjecture is somewhat non-standard. In particular, the best known algorithm to compute $\numocc{G}{\kmatch}$ takes time $\Omega\inparen{|V|^{k/2}}$
2022-05-02 08:10:58 -04:00
%(i.e. if this is the best algorithm then $c_0=\frac 14$)
2022-06-02 00:11:18 -04:00
%~\cite{k-match}.
2022-06-07 23:26:24 -04:00
The above result is saying is that, assuming Exponential Time Hypothesis (ETH), one can only hope for a slightly super-polynomial improvement over the trivial algorithm to compute $\numocc{G}{\kmatch}$.
%
2021-09-18 01:05:31 -04:00
2020-12-13 13:05:43 -05:00
Our hardness result in Section~\ref{sec:single-p} is based on the following conjectured hardness result:
2020-12-18 12:23:13 -05:00
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2020-12-13 13:05:43 -05:00
\begin{hypo}
\label{conj:graph}
2021-09-09 11:42:30 -04:00
There exists a constant $\eps_0>0$ such that given an undirected graph $G=(\vset,\edgeSet)$, computing $\numocc{G}{\tri}$ exactly cannot be done in time $o\inparen{|\edgeSet|^{1+\eps_0}}$.
2020-12-13 13:05:43 -05:00
\end{hypo}
2020-12-18 12:23:13 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
2021-09-17 19:46:07 -04:00
The so called {\em Triangle detection hypothesis} (cf.~\cite{triang-hard}), which states that detecting the presence of triangles in $G$ takes time $\Omega\inparen{|\edgeSet|^{4/3}}$, implies that in Conjecture~\ref{conj:graph} we can take $\eps_0\ge \frac 13$.
2020-12-13 13:05:43 -05:00
2021-09-15 17:15:53 -04:00
All of our hardness results rely on a simple lineage polynomial encoding of the edges of a graph.
To prove our hardness result, consider a graph $G=(\vset, \edgeSet)$, where $|\edgeSet| = m$, $\vset = [\numvar]$. Our lineage polynomial has a variable $X_i$ for every $i$ in $[\numvar]$.
2021-04-09 16:12:46 -04:00
Consider the polynomial
2021-06-11 11:22:58 -04:00
$\poly_{G}(\vct{X}) = \sum\limits_{(i, j) \in \edgeSet} X_i \cdot X_j.$
The hard polynomial for our problem will be a suitable power $k\ge 3$ of the polynomial above:
2020-12-18 12:23:13 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{Definition}\label{def:qk}
2021-09-02 17:01:17 -04:00
For any graph $G=(V,\edgeSet)$ and $\kElem\ge 1$, define
2021-09-15 17:15:53 -04:00
\[\poly_{G}^\kElem(X_1,\dots,X_n) = \left(\sum\limits_{(i, j) \in \edgeSet} X_i \cdot X_j\right)^\kElem.\]
2020-12-13 13:41:42 -05:00
\end{Definition}
2020-12-18 12:23:13 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2020-12-04 13:14:12 -05:00
2022-06-03 22:08:17 -04:00
\noindent Returning to \Cref{fig:two-step}, it can be seen that $\poly_{G}^\kElem(\vct{X})$ is the lineage polynomial from query $\qhard^k$, which we define next.
2022-06-02 00:11:18 -04:00
%Let us alias
%\begin{lstlisting}
%SELECT DISTINCT 1 FROM T $t_1$, R r, T $t_2$
%WHERE $t_1$.Point = r.Point$_1$ AND $t_2$.Point = %r.Point$_2$
%\end{lstlisting}
%as $Q^1$.
2022-06-03 22:08:17 -04:00
%The query $\qhard^k$ then becomes
\mdfdefinestyle{underbrace}{topline=false, rightline=false, bottomline=false, leftline=false, backgroundcolor=black!15!white, innerbottommargin=0pt}
\begin{mdframed}[style=underbrace]
2021-09-09 11:42:30 -04:00
\begin{lstlisting}
2022-06-03 14:13:41 -04:00
SELECT COUNT(*) FROM $\underbrace{Q_1\text{ JOIN }Q_1\text{ JOIN}\cdots\text{JOIN }Q_1}_{k\rm\ times}$
2021-09-09 11:42:30 -04:00
\end{lstlisting}
\end{mdframed}
2022-06-07 23:26:24 -04:00
In the above, $\query_1$ is as defined in \Cref{sec:intro}, which is the same as $\qhard^1$.
2022-06-03 22:08:17 -04:00
%
%\noindent %Consider again the \abbrCTIDB instance $\pdb$ of~\Cref{fig:two-step} and, for our hard instance, let $\bound = 1$. $\pdb$ generalizes to one compatible
2022-06-05 13:15:51 -04:00
We next define the instances for $T$ and $R$ that lead to the lineage polynomial in~\Cref{def:qk} as follows. Relation $T$ has $n$ tuples corresponding to each vertex for $i$ in $[n]$, each with probability $\prob$ and $R$ has tuples corresponding to the edges $\edgeSet$ (each with a probability of $1$).\footnote{Technically, $\poly_{G}^\kElem(\vct{X})$ should have variables corresponding to tuples in $R$ as well, but since they always are present with probability $1$, we drop those. Our argument also works when all the tuples in $R$ also are present with probability $\prob$ but to simplify notation we assign probability $1$ to edges.}
2022-06-07 23:26:24 -04:00
In other words, the \dbbaseName $\tupset$ contains the set of $\numvar$ unary tuples in $T$ (which corresponds to $\vset$) and $\numedge$ binary tuples in $R$ (which corresponds to $\edgeSet$).
Note that this implies that $\poly_{G}^\kElem$ is indeed a $1$-\abbrTIDB lineage polynomial.
2022-03-08 11:38:31 -05:00
Next, we note that the runtime for answering $\qhard^k$ on deterministic database $\tupset$, as defined above, is $O_k\inparen{\numedge}$ (i.e. deterministic query processing is `easy' for this query):
2021-09-14 08:21:57 -04:00
\begin{Lemma}\label{lem:tdet-om}
2022-06-03 22:08:17 -04:00
For $\qhard^k,\tupset$ as above,
2022-05-17 10:55:17 -04:00
$\qruntimenoopt{\qhard^k, \tupset, \bound}$ is $O_k\inparen{\numedge}$.
2021-09-14 08:21:57 -04:00
\end{Lemma}
2020-12-13 11:32:55 -05:00
\subsection{Multiple Distinct $\prob$ Values}
\label{sec:multiple-p}
2022-06-02 12:13:46 -04:00
We are now ready to present one of our main hardness result.
2020-12-18 12:23:13 -05:00
%
2021-04-09 16:12:46 -04:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
2020-12-13 14:16:32 -05:00
\begin{Theorem}\label{thm:mult-p-hard-result}
2022-06-07 23:26:24 -04:00
Let $\prob_0,\ldots,\prob_{2k}$ be $2k + 1$ distinct values in $(0, 1]$. Then computing $\rpoly_G^\kElem(\prob_i,\dots,\prob_i)$ (for all $i\in [2k+1]$) for arbitrary $G=(\vset,\edgeSet)$
2022-05-18 17:51:12 -04:00
needs time $\bigOmega{\kmatchtime}$, if $\kmatchtime\ge \omega\inparen{\abs{\edgeSet}}$.
2020-12-13 14:16:32 -05:00
\end{Theorem}
2020-12-18 12:23:13 -05:00
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
2022-06-02 12:13:46 -04:00
Note that the second (and third) row(s) of \Cref{tab:lbs} follow from %\Cref{prop:expection-of-polynom},
\Cref{thm:mult-p-hard-result}, \Cref{lem:tdet-om}, and \Cref{thm:k-match-hard} (\Cref{conj:known-algo-kmatch} resp.).
%\textcolor{red}{Need to put in a proof overview here-- Atri}
\Cref{thm:mult-p-hard-result} follows by observing that $\rpoly_G^\kElem(\prob,\dots,\prob)=\prob^{2k}\cdot \numocc{G}{\kmatch} +r(p)$, where $r(p)$ is a polynomial of degree at most $2k-1$ (with coefficients that just depend on $G$). By polynomial interpolation, knowing the values $\rpoly_G^\kElem(\prob_i,\dots,\prob_i)$ (over all $i\in [2k+1]$) allows us to compute all the coefficients, including $\numocc{G}{\kmatch}$.
%while the third row is proved by %\Cref{prop:expection-of-polynom}, \Cref{thm:mult-p-hard-result}, \Cref{lem:tdet-om}, and \Cref{conj:known-algo-kmatch}.
2022-06-02 00:11:18 -04:00
%Since \Cref{conj:known-algo-kmatch} is non-standard, the latter hardness result should be interpreted as follows. Any substantial polynomial improvement for \Cref{prob:bag-pdb-poly-expected} (over the trivial algorithm that converts $\poly$ into SMB and then uses \Cref{cor:expct-sop} for \abbrStepTwo) would lead to an improvement over the state of the art {\em upper} bounds on $\kmatchtime$. Finally,
2022-06-02 12:13:46 -04:00
2022-06-02 00:11:18 -04:00
Note that \Cref{thm:mult-p-hard-result} needs one to be able to compute the expected multiplicities over $(2k+1)$ distinct values of $p_i$, each of which corresponds to distinct $\bpd$ (for the same $\tupset$), which explain the `Multiple' entries in the second column of the second and third rows in \Cref{tab:lbs}. Next, we argue how to get rid of this latter requirement.
2022-06-02 12:13:46 -04:00
2021-04-10 14:35:38 -04:00
%%% Local Variables:
%%% mode: latex
%%% TeX-master: "main"
%%% End: