Moved definitions, lemmas, etc. to background/notation section.

This commit is contained in:
Aaron Huber 2020-12-11 20:19:45 -05:00
parent ba79c9ffd7
commit d9abe760a0
4 changed files with 50 additions and 51 deletions

View file

@ -1,43 +1,15 @@
%root: main.tex
\section{$1 \pm \epsilon$ Approximation Algorithm}
Since it is the case that computing the expected multiplicity of a compressed representation of a bag polynomial is hard, it is then desirable to have an algorithm to approximate the multiplicity in linear time, which is what we describe next.
\begin{Definition}[$\bi$~\cite{DBLP:series/synthesis/2011Suciu}]
A Block Independent Database ($\bi$) is a PDB whose tuples are partitioned in blocks, where we denote block $i$ as $\block_i$. Each $\block_i$ is independent of all other blocks, while all tuples sharing the same $\block_i$ are mutually exclusive.
\end{Definition}
A $\ti$ is also a $\bi$ where each tuple is its own block.
While the definition of polynomial $\poly(\vct{X})$ over a $\bi$ input doesn't change, we introduce an alternative notation which will come in handy. Given $\ell$ blocks, we write $\poly(\vct{X})$ = $\poly(X_{\block_1, 1},\ldots, X_{\block_1, \abs{\block_1}},$ $\ldots, X_{\block_\ell, \abs{\block_\ell}})$, where $\abs{\block_i}$ denotes the size of $\block_i$, and $\block_{i, j}$ denotes tuple $j$ residing in block $i$ for $j$ in $[\abs{\block_i}]$.
The number of tuples in the $\bi$ instance can be computed as $\numvar = \sum\limits_{i = 1}^{\ell}\abs{\block_i}$ .
When considering $\bi$ input, it becomes necessary to redefine $\rpoly(\vct{X})$.
\begin{Definition}[$\rpoly$ $\bi$ Redefinition]
A polynomial $\poly(\vct{X})$ over a $\bi$ instance is reduced to $\rpoly(\vct{X})$ with the following criteria. First, all exponents $e > 1$ are reduced to $e = 1$. Second, all monomials sharing the same $\block$ are dropped. Formally this is expressed as
\begin{equation*}
\rpoly(\vct{X}) = \poly(\vct{X}) \mod X_i^2 - X_i \mod X_{\block_s, t}X_{\block_s, u}
\end{equation*}
for all $i$ in $[\numvar]$ and for all $s$ in $\ell$, such that for all $t, u$ in $[\abs{block_s}]$, $t \neq u$.
\end{Definition}
We state the approximation algorithm in terms of a $\bi$.
First, let us introduce some useful definitions and notation. For illustrative purposes in the definitions below, let us consider when $\poly(\vct{X}) = 2x^2 + 3xy - 2y^2$.
\begin{Definition}[Degree]\label{def:degree}
The degree of polynomial $\poly(\vct{X})$ is the maximum sum of the exponents of a monomial, over all monomials.
\end{Definition}
The degree of $\poly(\vct{X})$ in the above example is $2$. In this paper we consider only finite degree polynomials.
\begin{Definition}[Expression Tree]\label{def:express-tree}
An expression tree $\etree$ is a binary %an ADT logically viewed as an n-ary
tree, whose internal nodes are from the set $\{+, \times\}$, with leaf nodes being either from the set $\mathbb{R}$ $(\tnum)$ or from the set of monomials $(\var)$. The members of $\etree$ are \type, \val, \vari{partial}, \vari{children}, and \vari{weight}, where \type is the type of value stored in the node $\etree$ (i.e. one of $\{+, \times, \var, \tnum\}$, \val is the value stored, and \vari{children} is the list of $\etree$'s children where $\etree_\lchild$ is the left child and $\etree_\rchild$ the right child. Remaining fields hold values whose semantics we will fix later. When $\etree$ is used as input of ~\cref{alg:mon-sam} and ~\cref{alg:one-pass}, the values of \vari{partial} and \vari{weight} will not be set. %SEMANTICS FOR \etree: \vari{partial} is the sum of $\etree$'s coefficients , n, and \vari{weight} is the probability of $\etree$ being sampled.
\end{Definition}
Note that $\etree$ need not encode an expression in the standard monomial basis. For example, instead of our running example, $\etree$ could represent a compressed form such as $(x + 2y)(2x - y)$.
Note that $\etree$ need not encode an expression in the standard monomial basis. For instance, $\etree$ could represent a compressed form of the running example, such as $(x + 2y)(2x - y)$.
\begin{Definition}[poly$(\cdot)$]\label{def:poly-func}
Denote $poly(\etree)$ to be the function that takes as input expression tree $\etree$ and outputs its corresponding polynomial. $poly(\cdot)$ is recursively defined on $\etree$ as follows, where $\etree_\lchild$ and $\etree_\rchild$ denote the left and right child of $\etree$ respectively.
@ -92,7 +64,7 @@ $\expandtree{\etree}$ is the pure sum of products expansion of $\etree$. The lo
\begin{Example}\label{example:expr-tree-T}
To illustrate with an example, consider the product $(x + 2y)(2x - y)$ and its expression tree $\etree$ in Figure ~\ref{fig:expr-tree-T}. The pure expansion of the product is $2x^2 - xy + 4xy - 2y^2 = \expandtree{\etree}$, logically viewed as $[(2, x^2), (-1, xy), (4, xy), (-2, y^2)]$.
To illustrate, consider the factorized representation $(x + 2y)(2x - y)$ of the running example. Its expression tree $\etree$ is illustrated in Figure ~\ref{fig:expr-tree-T}. The pure expansion of the product is $2x^2 - xy + 4xy - 2y^2 = \expandtree{\etree}$, logically viewed as $[(2, x^2), (-1, xy), (4, xy), (-2, y^2)]$.
\end{Example}
@ -138,7 +110,7 @@ To illustrate with an example, consider the product $(x + 2y)(2x - y)$ and its e
Let the positive tree, denoted $\abs{\etree}$ be the resulting expression tree such that, for each leaf node $\etree'$ of $\etree$ where $\etree'.\type$ is $\tnum$, $\etree'.\vari{value} = |\etree'.\vari{value}|$. %value $\coef$ of each coefficient leaf node in $\etree$ is set to %$\coef_i$ in $\etree$ is exchanged with its absolute value$|\coef|$.
\end{Definition}
Using the same polynomial from the above example, $poly(\abs{\etree}) = (x + 2y)(2x + y) = 2x^2 +xy +4xy + 2y^2 = 2x^2 + 5xy + 2y^2$. Note that this \textit{is not} the same as $\poly(\vct{X})$.
Using the same factorization from ~\cref{example:expr-tree-T}, $poly(\abs{\etree}) = (x + 2y)(2x + y) = 2x^2 +xy +4xy + 2y^2 = 2x^2 + 5xy + 2y^2$. Note that this \textit{is not} the same as $\poly(\vct{X})$.
\begin{Definition}[Evaluation]\label{def:exp-poly-eval}
Given an expression tree $\etree$ and $\vct{v} \in \mathbb{R}^\numvar$, $\etree(\vct{v}) = poly(\etree)(\vct{v})$.
@ -158,7 +130,7 @@ For any query polynomial $\poly(\vct{X})$, an approximation of $\rpoly(\prob_1,\
\end{Theorem}
\subsection{Approximating $\rpoly$}
We state the approximation algorithm in terms of a $\bi$.
\subsubsection{Description}
Algorithm ~\ref{alg:mon-sam} approximates $\rpoly$ using the following steps. First, a call to $\onepass$ on its input $\etree$ produces a non-biased weight distribution over the monomials of $\expandtree{\etree}$ and a correct count of $|\etree|(1,\ldots, 1)$, i.e., the number of monomials in $\expandtree{\etree}$. Next, ~\cref{alg:mon-sam} calls $\sampmon$ to sample one monomial and its sign from $\expandtree{\etree}$. The sampling is repeated $\ceil{\frac{2\log{\frac{2}{\delta}}}{\epsilon^2}}$ times, where each of the samples are evaluated with input $\vct{p}$, multiplied by $1 \times sign$, and summed. The final result is scaled accordingly returning an estimate of $\rpoly$ with the claimed $(\error, \conf)$-bound of ~\cref{lem:mon-samp}.

View file

@ -1,6 +1,6 @@
%root:main.tex
\subsection{Multiple Distinct $\prob$ Values}
\section{Multiple Distinct $\prob$ Values}
We would like to argue for a compressed version of $\poly(\vct{w})$, in general $\expct_{\vct{w}}\pbox{\poly(\vct{w})}$ cannot be computed in linear time.
\AR{Added the hardness result below.}

View file

@ -1,11 +1,11 @@
%root: main.tex
%!TEX root = ./main.tex
%\onecolumn
\section{Polynomial Formulation and Equivalences}
\subsection{Polynomial Formulation and Equivalences}
Before proceeding, note that the following is assuming $\ti$s in the setting of \textit{bag} semantics.
Before proceeding, note that the following is assuming $\ti/\bi$ (set) input with the output in the bag setting.
Throughout the note, we also make the following \textit{assumption}.
Let us use the expression $(x + y)^2$ for a running example in the following definitions.
\begin{Definition}[Monomial]\label{def:monomial}
A monomial is a product of a fixed set of variables, each raised to a non-negative integer power.
@ -17,14 +17,23 @@ For the term $2xy$, by ~\cref{def:monomial} the monomial is $xy$.
A polynomial is in standard monomial basis when it is fully expanded out such that no product of sums exist and where each unique monomial appears exactly once.
\end{Definition}
For example, consider the expression $(x + y)^2$. The standard monomial basis for this expression is $x^2 +2xy + y^2$. While $x^2 + xy + xy + y^2$ is an expanded form of the expression, it is not the standard monomial basis since $xy$ appears more than once.
The standard monomial basis for the running example is $x^2 +2xy + y^2$. While $x^2 + xy + xy + y^2$ is an expanded form of the expression, it is not the standard monomial basis since $xy$ appears more than once.
Throughout this paper, we also make the following \textit{assumption}.
\begin{Assumption}\label{assump:poly-smb}
All polynomials considered are in standard monomial basis, i.e., $\poly(\vct{X}) = \sum\limits_{\vct{d} \in \mathbb{N}^\numvar}q_d \cdot \prod\limits_{i = 1, d_i \geq 1}^{\numvar}X_i^{d_i}$, where $q_d$ is the coefficient for the monomial encoded in $\vct{d}$ and $d_i$ is the $i^{th}$ element of $\vct{d}$.
\end{Assumption}
While the definition of polynomial $\poly(\vct{X})$ over a $\bi$ input doesn't change, we introduce an alternative notation which will come in handy. Given $\ell$ blocks, we write $\poly(\vct{X})$ = $\poly(X_{\block_1, 1},\ldots, X_{\block_1, \abs{\block_1}},$ $\ldots, X_{\block_\ell, \abs{\block_\ell}})$, where $\abs{\block_i}$ denotes the size of $\block_i$, and $\block_{i, j}$ denotes tuple $j$ residing in block $i$ for $j$ in $[\abs{\block_i}]$.
The number of tuples in the $\bi$ instance can be (trivially) computed as $\numvar = \sum\limits_{i = 1}^{\ell}\abs{\block_i}$ .
\begin{Definition}\label{def:qtilde}
\begin{Definition}[Degree]\label{def:degree}
The degree of polynomial $\poly(\vct{X})$ is the maximum sum of the exponents of a monomial, over all monomials when $\poly(\vct{X})$ is in SOP form.
\end{Definition}
The degree of the running example is $2$. In this paper we consider only finite degree polynomials.
\begin{Definition}[$\rpoly(\vct{X})$] \label{def:qtilde}
Define $\rpoly(X_1,\ldots, X_\numvar)$ as the reduced version of $\poly(X_1,\ldots, X_\numvar)$, of the form
$\rpoly(X_1,\ldots, X_\numvar) = $
@ -40,10 +49,21 @@ Consider when $\poly(x, y) = (x + y)(x + y)$. Then the expanded derivation for
\end{align*}
\end{Example}
Intuitively, $\rpoly(\textbf{X})$ is the expanded sum of products form of $\poly(\textbf{X})$ such that if any $X_j$ term has an exponent $e > 1$, it is reduced to $1$, i.e. $X_j^e\mapsto X_j$ for any $e > 1$.
Alternatively, one can gain intuition for $\rpoly$ by thinking of $\rpoly$ as the resulting sum of product expansion of $\poly$ when $\poly$ is in a factorized form such that none of its terms have an exponent $e > 1$, if the product operator is idempotent.
Intuitively, $\rpoly(\textbf{X})$ is the SOP form of $\poly(\textbf{X})$ such that if any $X_j$ term has an exponent $e > 1$, it is reduced to $1$, i.e. $X_j^e\mapsto X_j$ for any $e > 1$.
Alternatively, one can gain intuition for $\rpoly$ by thinking of $\rpoly$ as the resulting SOP of $\poly(\vct{X})$ with an idemptent product operator.
The usefulness of this reduction will be seen shortly.
When considering $\bi$ input, it becomes necessary to redefine $\rpoly(\vct{X})$.
\begin{Definition}[$\rpoly$ $\bi$ Redefinition]
A polynomial $\poly(\vct{X})$ over a $\bi$ instance is reduced to $\rpoly(\vct{X})$ with the following criteria. First, all exponents $e > 1$ are reduced to $e = 1$. Second, all monomials sharing the same $\block$ are dropped. Formally this is expressed as
\begin{equation*}
\rpoly(\vct{X}) = \poly(\vct{X}) \mod X_i^2 - X_i \mod X_{\block_s, t}X_{\block_s, u}
\end{equation*}
for all $i$ in $[\numvar]$ and for all $s$ in $\ell$, such that for all $t, u$ in $[\abs{block_s}]$, $t \neq u$.
\end{Definition}
The usefulness of this reduction will be seen in ~\cref{lem:exp-poly-rpoly}.
\begin{Lemma}\label{lem:pre-poly-rpoly}
When $\poly(X_1,\ldots, X_\numvar) = \sum\limits_{\vct{d} \in \{0,\ldots, B\}^\numvar}q_{\vct{d}} \cdot \prod\limits_{\substack{i = 1\\s.t. d_i\geq 1}}^{\numvar}X_i^{d_i}$, we have then that $\rpoly(X_1,\ldots, X_\numvar) = \sum\limits_{\vct{d} \in \{0,\ldots, B\}^\numvar} q_{\vct{d}}\cdot\prod\limits_{\substack{i = 1\\s.t. d_i\geq 1}}^{\numvar}X_i$.
@ -67,7 +87,7 @@ Note that any $\poly$ in factorized form is equivalent to its sum of product exp
Define all variables $X_i$ in $\poly$ to be independent.
\begin{Lemma}\label{lem:exp-poly-rpoly}
The expectation over possible worlds in $\poly$ is equal to $\rpoly(\prob_1,\ldots, \prob_\numvar)$.
The expectation over possible worlds in $\poly(\vct{X})$ is equal to $\rpoly(\prob_1,\ldots, \prob_\numvar)$.
\begin{equation*}
\expct_{\vct{w}}\pbox{\poly(\vct{w})} = \rpoly(\prob_1,\ldots, \prob_\numvar).
\end{equation*}

View file

@ -1,10 +1,10 @@
%root: main.tex
%!TEX root=./main.tex
%\onecolumn
\section{Query translation into polynomials}
\section{Background Knowledge and Notation}
\subsection{Introduction}
\subsection{PDBs}
An incomplete database $\idb$ is a set of deterministic databases $\db$ where each element is known as a possible world.
@ -13,21 +13,28 @@ Denote the schema of $\db$ as $\sch(\db)$. When $\idb$ is a probabilistic datab
The possible worlds semantics gives a framework for how to think about running queries over $\idb$. Given a query $\query$, $\query$ is deterministically run over each $\db \in \idb$, and the output of $\query(\idb)$ is defined as the set of results (worlds) from running $\query$ over each $\db_i \in \idb$. We write this formally as,
\[\query(\idb) = \comprehension{\query(\db)}{\db \in \idb}.\]
\begin{Definition}[$\bi$~\cite{DBLP:series/synthesis/2011Suciu}]
A Block Independent Database ($\bi$) is a PDB whose tuples are partitioned in blocks, where we denote block $i$ as $\block_i$. Each $\block_i$ is independent of all other blocks, while all tuples sharing the same $\block_i$ are mutually exclusive.
\end{Definition}
\begin{Definition}[$\ti$]
A Tuple Independent Database ($\ti$) is a special case of a $\bi$ such that each tuple is its own block.
\end{Definition}
\subsection{Modeling and Semantics}
Define $\vct{X}$ to be the variables $X_1,\dots,X_M$. We emphasize that formal variables do not have a fixed domain type prior to assignment. Let the set of all tuples in domain of $\sch(\db)$ be $\tset$.
Define $\vct{X}$ to be the vector of variables $X_1,\dots,X_M$. Let the set of all tuples in domain of $\sch(\db)$ be $\tset$.
\subsubsection{K-relations}\label{subsubsec:k-rel}
A K-relation~\cite{DBLP:conf/pods/GreenKT07} is a relation whose tuples are each annotated with an expression whose values come from a commutative K-semiring, denoted $\{K, \oplus, \otimes, \mathbbold{0}, \mathbbold{1}\}$. A commutative $K$-semiring has associative and commutative operators $\oplus$ and $\otimes$, with $\otimes$ distributing over $\oplus$, $\mathbbold{0}$ the identity of $\oplus$, $\mathbbold{1}$ likewise of $\otimes$, and element $\mathbbold{0}$ anihilates all elements of $K$ when being combined with $\otimes$. The information encoded in the annotation depends on the underlying semiring of the relation.
As noted in \cite{DBLP:conf/pods/GreenKT07}, the $\mathbb{N}[\vct{X}]$-semiring is a semiring over the set $\mathbb{N}[\vct{X}]$ of all polynomials, whose variables can then be substituted with $K$-values from other semirings, evaluating the operators with the operators of the substituted semiring, to produce varying semantics such as set, bag, and security annotations.
As noted in \cite{DBLP:conf/pods/GreenKT07}, the $\mathbb{N}[\vct{X}]$-semiring is a semiring over the set $\mathbb{N}[\vct{X}]$ of all polynomials, whose variables can then be substituted with $K$-values from other semirings, evaluating the operators with the operators of the substituted semiring, to produce varying semantics such as set, bag, and security.
Further define $\nxdb$ as an $\mathbb{N}[\vct{X}]$ database where each tuple $\tup \in \db$ is annotated with a polynomial over variables $X_1,\ldots, X_M$.
Since $\nxdb$ is a database that maps tuples to polynomials, it is customary for arbitrary table $\rel$ to be viewed as a function $\rel: \tset \mapsto \mathbb{N}[\vct{X}]$, where $\rel(\tup)$ denotes the polynomial annotating tuple $\tup$.
It has been shown in previous work that commutative semirings precisely model translations of RA+ query operations to $K$-annotations.
The evalution semantics notation $\llbracket \cdot \rrbracket = x$ simply mean that the result of evaluating expression $\cdot$ is given by following the semantics $x$. Given a query $\query$, operations in $\query$ are translated into the following polynomial expressions.
%The evalution semantics notation $\llbracket \cdot \rrbracket = x$ simply mean that the result of evaluating expression $\cdot$ is given by following the semantics $x$.
Given a query $\query$, operations in $\query$ are translated into the following polynomial expressions.
\begin{align*}
&\eval{\project_A(\rel)}(\tup)&& = &&\sum_{\tup': \project_A(\tup) = \tup} \eval{\rel}(\tup')\\
@ -40,13 +47,13 @@ The evalution semantics notation $\llbracket \cdot \rrbracket = x$ simply mean t
&\eval{R}(\tup) && = &&\rel(\tup)
\end{align*}
The above semantics show us how to obtain the $K$-annotation on a tuple in the result of query $\query$ from the annotations on the tuples in the input of $\query$. When used with $\mathbb B$-typed variables, an $\mathbb{N}[\vct{X}]$ relation is effectively a C-Table \cite{DBLP:conf/pods/GreenKT07}, since all first order formulas can be equivalently modeled by polynomials, where $\oplus$ is disjunction and $\otimes$ is conjunction.
Using $\mathbb B$-typed variables in an $\mathbb{N}[\vct{X}]$ relation would correspond to substituting values and operators from the $\{\mathbb{B}, \vee, \wedge, \bot, \top\}$ semiring. In like manner, when using variables from the $\mathbb{N}$ domain, the annotations then effectively model bag semantics, where the variables and $\oplus$ and $\otimes$ operations come from the natural numbers semiring $\{\mathbb{N}, +, \times, 0, 1\}$.
The above semantics show us how to obtain the $K$-annotation on a tuple in the result of query $\query$ from the annotations of the input tuples. When used with $\mathbb B$-typed variables, an $\mathbb{N}[\vct{X}]$ relation is effectively a C-Table \cite{DBLP:conf/pods/GreenKT07}, since all first order formulas can be equivalently modeled by polynomials, where $\oplus$ is disjunction and $\otimes$ is conjunction.
This is the equivalent to substituting values and operators from the $\{\mathbb{B}, \vee, \wedge, \bot, \top\}$ semiring. In like manner, when assigning values from the $\mathbb{N}$ domain, the polynomials then model bag semantics, where the variables and $\oplus$ and $\otimes$ operations come from the natural numbers semiring $\{\mathbb{N}, +, \times, 0, 1\}$.
\subsection{Defining the Data}\label{subsec:def-data}
For the set of possible worlds, $\wSet$, i.e. the set of all $\db_i \in \idb$, define an injective mapping to the set $\{0, 1\}^M$, where for each vector $\vct{w} \in \{0, 1\}^M$ there is at most one element $\db_i \in \idb$ mapped to $\vct{w}$.
In the general case, the binary value of $\vct{w}$ uniquely identifies a potential possible world. For example, consider the case of the Tuple Independent Database $(\ti)$ data model in which each table is a set of tuples, each of which is independent of one another, and individually occur with a specific probability $\prob_\tup$. Because of independence, a $\ti$ with $\numvar$ tuples naturally has $2^\numvar$ possible worlds, thus $\numvar = M$, and the injective mapping for each $\vct{w} \in \{0, 1\}^M$ is trivial. In the Block Independent Disjoint data model ($\bi$), because of the disjoint condition on tuples within the same block, a $\bi$ may not have exactly $2^M$ possible worlds since there are combinations of tuples that cannot exist in the encoding.
Denote a random variable selecting a world according to distribution $P$ to be $\rw$. Provided that for any non-possible world $\vct{w} \in \{0, 1\}^M, \pd[\rw = \vct{w}] = 0$, a probability distribution over $\{0, 1\}^M$ is a distribution over $\Omega$, which we have already defined as $\pd$.
Denote a random variable selecting a world according to distribution $P$ to be $\rw$. Provided that for any non-possible world $\vct{w} \in \{0, 1\}^M, \pd[\rw = \vct{w}] = 0$, a probability distribution over $\{0, 1\}^M$ is a distribution over $\Omega$.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%This could be a way to think of world binary vectors in the general case
@ -63,6 +70,6 @@ One of the aggregates we desire to compute over the annotated polynomial is the
Above, $\poly(\vct{w})$ is used to mean the assignment of $\vct{w}$ to $\vct{X}$.
For a $\ti$, the bit-string world value $\vct{w}$ can be used as indexing to determine which tuples are present in the $\vct{w}$ world, where the $i^{th}$ bit position $(\wbit_i)$ represents whether a tuple $\tup_i$ appears in the unique world identified by the binary value of $\vct{w}[i]$. Denote the vector $\vct{p}$ to be a vector whose elements are the individual probabilities $\prob_i$ of each tuple $\tup_i$ such that those probabilities produce the possible worlds in D with a distribution $\pd$ over all worlds. Let $\pd^{(\vct{p})}$ represent the distribution induced by $\vct{p}$.
For a $\ti$, the bit-string world value $\vct{w}$ can be used as indexing to determine which tuples are present in the $\vct{w}$ world, where the $i^{th}$ bit position $(\wbit_i)$ represents whether a tuple $\tup_i$ appears in the unique world $\vct{w}$. Denote the vector $\vct{p}$ to be a vector whose elements are the individual probabilities $\prob_i$ of each tuple $\tup_i$ such that those probabilities produce the possible worlds in D with a distribution $\pd$ over all worlds. Let $\pd^{(\vct{p})}$ represent the distribution induced by $\vct{p}$.
\[\expct_{\rw\sim \pd^{(\vct{p})}}\pbox{\poly(\rw)} = \sum\limits_{\vct{w} \in \{0, 1\}^\numvar} \poly(\vct{w})\prod_{\substack{i \in [\numvar]\\ s.t. \wElem_i = 1}}\prob_i \prod_{\substack{i \in [\numvar]\\s.t. w_i = 0}}\left(1 - \prob_i\right).\]