minor formatting tweaks, some flow tweaks, and adding a conclusions section.

paper.tex

@@ -60,9 +60,9 @@ Website: \texttt{http://odin.cse.buffalo.edu/research/}
 \label{sec:pocketdata}
 \input{sections/6-pocketdata}
 
-% \section{Conclusions and Future Work}
-% \label{sec:conc}
-% \input{sections/7-conclusions}
+\section{Conclusions}
+\label{sec:conc}
+\input{sections/7-conclusions}
 
 
 \bibliographystyle{plain}

sections/1-introduction.tex

@@ -2,20 +2,20 @@
 Since the introduction of the smartphone, mobile computing has become pervasive in our society, with one in every five people in the world owning a smartphone~\cite{phones}. Mobile devices, however,
 have evolved far beyond the stereotypical smartphone and tablet, and are now employed in a wide variety of domains. Of the currently available mobile systems, Android has seen the most widespread deployment outside of the consumer electronics market. Android's open source nature has prompted its ubiquitous adoption in sensing, medical, command and control, robotics, and automotive applications.
 
-For each of these domains, Android supports specific client-side applications that often utilize an application-resident, or \textit{embedded database} called SQLite~\cite{sqlite} for persistent state and data analysis. Even Android
+For each of these domains, Android supports specific client-side applications that often utilize an application-resident, or \textit{embedded database} called SQLite~\cite{sqlite} for tasks such as persisting state. Even Android
 itself, specifically the framework layer, makes use of embedded databases.
 The way in which mobile applications use databases, is however, rather different from traditional server farms and desktop applications that focus on ``big data.''
 Based on the experiments we present in this paper, SQLite on \textit{one} average Android smart-phone satisfies over 178 thousand database requests \textit{per day}, or about 2 requests every second. 
 As such, client-side mobile applications represent a new and rapidly growing class of \textit{pocket-scale} database consumers. The execution context for these client-side application workloads varies wildly. This means that the performance of the database is not only dependent on the query workload being executed by the client-side application, but is also impacted by other applications that may have access to the database, other applications currently using system resources ({\em e.g.} processor, memory,
 file system), and the overall energy available and power profile of the system itself ({\em e.g.} voltage scaling, power governors, OS policies).
 
-The challenges that arise in handling pocket-scale data --- that is, minimizing power consumption, latency, and io --- are familiar ground for database practitioners, but the trade-offs and workload characteristics are far less well understood. In this paper, we present the results of a one-month trace of SQLite activity on 11 Android smartphones, part of a real-world experimental testbed called PhoneLab~\cite{phonelab}.  We show that the workload characteristics encountered by SQLite on these phones differ substantially from the characteristics of database workloads expressed in popular database benchmarking suites.  We argue that a new benchmark for mobile and embedded databases could spur innovation in this area, and outline the workload characteristics of such a benchmark.  
+The challenges that arise in handling pocket-scale data --- that is, minimizing power consumption, latency, and I/O --- are familiar ground for database practitioners, but the trade-offs and workload characteristics are far less well understood. In this paper, we present the results of a one-month trace of SQLite activity on 11 Android smartphones, part of a real-world experimental testbed called \PhoneLab{}~\cite{phonelab}.  We show that the workload characteristics encountered by SQLite on these phones differ substantially from the characteristics of database workloads expressed in popular database benchmarking suites.  We argue that a new benchmark for mobile and embedded databases could spur innovation in this area, and outline the workload characteristics of such a benchmark.  
 
 \noindent The contributions of this paper are the following:
 \begin{itemize}
 \item A detailed examination of SQLite~\cite{sqlite} usage in a real-world mobile context. We compare our gathered results on mobile application and Android framework usage of SQLite to tradition TPC benchmark
 workloads. 
-\item The synthesis of an open source data set gathered over one month of "in the wild" execution of database queries using eleven test subjects and their personal smartphones. Data was gathered using PhoneLab~\cite{phonelab}.
+\item The synthesis of an open source data set gathered over one month of "in the wild" execution of database queries using eleven test subjects and their personal smartphones. Data was gathered using \PhoneLab{}~\cite{phonelab}.
 \item An outline of workload characteristics and other desiderata for a proposed TPC-MOBILE benchmark.
 \end{itemize}
 

sections/2-overview.tex

@@ -3,10 +3,10 @@ Our primary observation is that an embedded database workload in a modern mobile
 
 Many of these workload characteristics are motivated by factors unique to embedded databases.  For example, SQLite uses single-file databases that have a standard, platform-independent format.  As a consequence, it is common to see entire databases, indexes and all, transported in their entirety through web downloads or as attachments to other files~\cite{Dit2015CIDR}.  A common pattern we observed was for a cloud service to package a fragment of its state into a SQLite database, which could then be cached locally on the device for lower-latency and offline access.
 
-Optimization targets also differ substantially.  Latency is a primary concern, but at vastly different scales.  Over our one-month trial, the average SQL statement took 2 ms to evaluate, and even complex \texttt{SELECT} queries with 4-level deep nesting only took an average of 120 ms. %Flash wear~\cite{SIGMOD2014?} is also a concern.
+Query optimization goals also differ substantially. For example, latency is a primary concern, but at vastly different scales.  Over our one-month trial, the average SQL statement took 2 ms to evaluate, and even complex \texttt{SELECT} queries with 4-level deep nesting only took an average of 120 ms. 
 
 Finally, unlike typical server-class benchmark workloads where throughput is a key factor, embedded databases have fixed, ``small data"~\cite{Dit2015CIDR}  workloads and need to share computing resources fairly with other processes on the same device.  This means that in stark contrast to server-class workloads, the database is idle more frequently.  Periods of low-utilization are opportunities for background optimization, but must be managed against the needs of other applications running on the device, as well as the device's limited power budget. We use the term ``pocket data" to refer to data management settings that exhibit such characteristics.
 
-Pocket data workloads represent a growing, and extremely important class of database consumers.  Unfortunately, research and development on embedded databases (\textit{e.g.},~\cite{jeong2013iostack,kang2013xftl}) is presently obligated to rely on micro-benchmarks or anecdotal observations about the needs and requirements of embedded database engines.  In this paper, we lay out the characteristics of a one month trace of SQLite operations performed on eleven Android smartphones participating in the PhoneLab experimental platform~\cite{phonelab}.  We believe that a new TPC-MOBILE benchmark that captures these characteristics can provide a principled, standardized way to evaluate advances in mobile database technology, which will in turn, help to drive the development of such advances.
+Pocket data workloads represent a growing, and extremely important class of database consumers.  Unfortunately, research and development on embedded databases (\textit{e.g.},~\cite{jeong2013iostack,kang2013xftl}) is presently obligated to rely on micro-benchmarks or anecdotal observations about the needs and requirements of embedded database engines.  In this paper, we lay out the characteristics of a one month trace of SQLite operations performed on eleven Android smartphones participating in the \PhoneLab{} experimental platform~\cite{phonelab}.  We believe that a new TPC-MOBILE benchmark that captures these characteristics can provide a principled, standardized way to evaluate advances in mobile database technology, which will in turn, help to drive the development of such advances.
 
 %%  LocalWords:  OLTP OLAP SQLite ms Android smartphones PhoneLab TPC

sections/4-queryc.tex

@@ -68,14 +68,13 @@ Figure~\ref{fig:topBottom10Apps} shows the 10 most frequent and 10 least frequen
 \label{fig:coarseSelectComplexity}
 \end{figure*}
 
-Of the 45 million queries analyzed, 33.47 million were read-only SELECT queries.  
+Of the 45 million queries analyzed, 33.47 million were read-only \texttt{SELECT} queries.  
 Figure~\ref{fig:coarseSelectComplexity} shows the distribution of \texttt{SELECT} queries by number of tables accessed by the query, as well as the maximum level of query nesting.  Nesting includes from-nesting (\textit{e.g.}, \texttt{SELECT \ldots\ FROM (SELECT \ldots)}), as well as expression-nesting (\textit{e.g.}, \texttt{SELECT \ldots\ WHERE EXISTS (SELECT \ldots)}).  
 Even at this coarse-grained view of query complexity, the read-only portion of the embedded workload distinguishes itself from existing TPC benchmarks.  
 
 Like TPC-C~\cite{tpcc}, the vast majority of the workload involves simple, small requests for data that touch a small number of tables.
-29.15 million, or about 87 percent of the \texttt{SELECT} queries were simple select-project-join queries.  Of those, 28.72 million or about 86 percent of all queries were simple single-table scans or look-ups.  In these queries, which form the bulk of SQLite's read workload, the query engine exists simply to provide an iterator over the relationally structured data it is being used to store.  
-
-Conversely, the workload also has a tail that consists of complex, TPC-H-like~\cite{tpch} queries.  Several hundred thousand queries involve at least 2 levels of nesting, and over a hundred thousand queries access 5 or more tables.  In 10 instances of a similar query, attributable to the Google Play Games Service\footnote{\url{https://developers.google.com/games/services/}}, a single \texttt{SELECT} query accesses as many as 8 distinct tables to combine together developer-provided game state, user preferences, device profile meta-data, and historical game-play results from the user.
+29.15 million, or about 87\% of the \texttt{SELECT} queries were simple select-project-join queries.  Of those, 28.72 million or about 86\% of all queries were simple single-table scans or look-ups.  In these queries, which form the bulk of SQLite's read workload, the query engine exists simply to provide an iterator over the relationally structured data it is being used to store.  
+Conversely, the workload also has a tail that consists of complex, TPC-H-like~\cite{tpch} queries.  Several hundred thousand queries involve at least 2 levels of nesting, and over a hundred thousand queries access 5 or more tables.  As an extreme example, our trace includes 10 similar \texttt{SELECT} queries issued by the Google Play Games Service\footnote{\url{https://developers.google.com/games/services/}}, each of which accesses up to 8 distinct tables to combine developer-provided game state, user preferences, device profile meta-data, and historical game-play results from the user.
 
 \subsubsection{Simple \texttt{SELECT} Queries}
 
@@ -91,7 +90,7 @@ Figure~\ref{fig:spjsByWidthAndWhere} shows queries of this class, broken down by
 \texttt{SELECT R.A FROM R, S WHERE R.B = S.B AND S.C = 10}
 This query would have a join width of 2 (\texttt{R}, \texttt{S}) and 2 conjunctive terms (\texttt{R.B = S.B} and \texttt{S.C = 10}).  For uniformity, \texttt{NATURAL JOIN} and \texttt{JOIN ON} (\textit{e.g.}, \texttt{SELECT R.A from R JOIN S ON B}) expressions appearing in the \texttt{FROM} clause are rewritten into equivalent expressions in the \texttt{WHERE} clause.
 
-The first column of this table indicates queries to a single relation.  Just over 1 million queries were full table scans (0 where clauses), and just under 27 million queries involved only a single conjunctive term.  This latter class constitutes the bulk of the simple query workload, at just over 87 percent of the simple look-up queries.  Single-clause queries appear to be the norm.  Recall that an N-way equi-join requires N-1 conjunctive terms; Spikes occur in the number of queries with one more term than strictly required to perform a join, suggesting a constraint on at least one relation.
+The first column of this table indicates queries to a single relation.  Just over 1 million queries were full table scans (0 where clauses), and just under 27 million queries involved only a single conjunctive term.  This latter class constitutes the bulk of the simple query workload, at just over 87\% of the simple look-up queries.  Single-clause queries appear to be the norm.  Recall that an N-way equi-join requires N-1 conjunctive terms; Spikes occur in the number of queries with one more term than strictly required to perform a join, suggesting a constraint on at least one relation.
 
 \begin{figure}
 \centering
@@ -102,7 +101,7 @@ The first column of this table indicates queries to a single relation.  Just ove
 \label{fig:singleClauseExpressions}
 \end{figure} 
 
-Narrowing further, we examine simple look-up queries referencing only a single source table and a single conjunctive term in the WHERE clause.  Figure~\ref{fig:singleClauseExpressions} summarizes the structure of the predicate that appears in each of these queries.  In this figure, constant terms (Const) are any primitive value term (\textit{e.g.}, a quoted string, an integer, or a float), or any JDBC-style parameter ($?$).  For simple relational comparators, we group together \textit{in}equalities (\textit{i.e.}, $<$, $\leq$, $>$, $\geq$ and $\neq$) under the symbol $\theta$, and explicitly list equalities.  Other relational operators such as \texttt{LIKE}, \texttt{BETWEEN}, and \texttt{IN} are also seen with some frequency.  However, the majority of look-ups (85\% of all simple look-ups) are exact match look-ups.  
+Narrowing further, we examine simple look-up queries referencing only a single source table and a single conjunctive term in the WHERE clause.  Figure~\ref{fig:singleClauseExpressions} summarizes the structure of the predicate that appears in each of these queries.  In this figure, constant terms (Const) are any primitive value term (\textit{e.g.}, a quoted string, an integer, or a float), or any JDBC-style parameter ($?$).  For simple relational comparators, we group together \textit{in}equalities (\textit{i.e.}, $<$, $\leq$, $>$, $\geq$ and $\neq$) under the symbol $\theta$, and explicitly list equalities.  Other relational operators such as \texttt{LIKE}, \texttt{BETWEEN}, and \texttt{IN} are also seen with some frequency.  However, the majority (85\% of all simple look-ups) are exact match look-ups.  
 Not surprisingly, this suggests that the most common use-case for SQLite is as a relational key-value store.  As we show shortly through a per-app analysis of the data (Section~\ref{sec:select:perapp}), 24 out of the 179 apps that we encountered posed no queries other than exact look-ups and full table scans.  
 
 \subsubsection{Other \texttt{SELECT} Queries}
@@ -143,14 +142,16 @@ The most commonly used function was \texttt{GROUP\_CONCAT}, an aggregate operato
 \label{sec:select:perapp}
 
 \begin{figure*}
-\centering
+  \vspace*{-0.3in}
   \begin{subfigure}[t]{0.5\textwidth}
-  	\includegraphics[width=\textwidth]{graphs/select_count_cdf_by_app}
+  	\centering
+  	\includegraphics[width=0.9\textwidth]{graphs/select_count_cdf_by_app}
     \caption{}
     \label{fig:selectByApp:all}
   \end{subfigure}%
   \begin{subfigure}[t]{0.5\textwidth}
-	\includegraphics[width=\textwidth]{graphs/select_percent_simple_cdf_by_app} 
+  	\centering
+	\includegraphics[width=0.9\textwidth]{graphs/select_percent_simple_cdf_by_app} 
     \caption{}
     \label{fig:selectByApp:simple}
   \end{subfigure}%
@@ -162,7 +163,7 @@ We next break the \texttt{SELECT} workload down by the calling application (app)
 Due to limitations of the logging infrastructure, 4.32 million queries (just over 12.9\% of the workload) could not be associated with a specific application, and our app-specific analysis excludes these queries.
 Additionally, system services in Android are often implemented as independent apps and counted as such in the numbers presented.
 
-Over the course of the one-month trace we observed 179 distinct apps,, varying from built-in Android applications such as \textit{Gmail} or \textit{YouTube} to video players such as \textit{VLC} to games such as \textit{3 Kingdoms}.  Figure~\ref{fig:selectByApp:all} shows the cumulative distribution of apps sorted by the number of queries that the app performs.  The results are highly skewed, with the top 10\% of apps each posing more than 100 thousand queries over the one month trace, or an average of about 1 query every 4 minutes on any given phone.  The most query-intensive system service, \textit{Media Storage} was responsible for 13.57 million queries or just shy of 40 queries per minute per phone.  The most query-intensive user-facing app was \textit{Google+}, which performed 1.94 million queries over the course of the month or 5 queries per minute.
+Over the course of the one-month trace we observed 179 distinct apps, varying from built-in Android applications such as \textit{Gmail} or \textit{YouTube}, to video players such as \textit{VLC}, to games such as \textit{3 Kingdoms}.  Figure~\ref{fig:selectByApp:all} shows the cumulative distribution of apps sorted by the number of queries that the app performs.  The results are extremely skewed, with the top 10\% of apps each posing more than 100 thousand queries over the one month trace.  The most query-intensive system service, \textit{Media Storage} was responsible for 13.57 million queries or just shy of 40 queries per minute per phone.  The most query-intensive user-facing app was \textit{Google+}, which performed 1.94 million queries over the course of the month or 5 queries per minute.
 At the other end of the spectrum, the bottom 10\% of apps posed as few as 30 queries over the entire month.
 
 
@@ -195,8 +196,8 @@ Write statements, \texttt{INSERT}, \texttt{INSERT OR REPLACE} (here abbreviated
 \label{fig:allDeleteConditionBreakdown}
 \end{figure} 
 
-The trace includes 1.25 million \texttt{DELETE} statements.  This was by far the most expensive class of statement, with an average \texttt{DELETE} taking just under 4 ms to complete.  A significant portion of this cost is attributable to the use of \texttt{DELETE} as a form of bulk erasure.  323 thousand \texttt{DELETE}s have no exact match condition in their WHERE clause, while 528 thousand do include a range predicate.  
-\texttt{DELETE} predicates can become quite complex; 46,122 \texttt{DELETE}s (just under 3.7 percent) use nested \texttt{SELECT} queries, and touch as many as 7 separate tables (in 616 cases).  
+The trace includes 1.25 million \texttt{DELETE} statements.  This was by far the most expensive class of statement, with an average \texttt{DELETE} taking just under 4 ms to complete.  A significant portion of this cost is attributable to the use of \texttt{DELETE} as a form of bulk erasure.  As shown in Figure~\ref{fig:allDeleteConditionBreakdown}, 323 thousand \texttt{DELETE}s have no exact match condition in their WHERE clause, while 528 thousand do include a range predicate.  
+\texttt{DELETE} predicates can become quite complex; 46,122 \texttt{DELETE}s (just under 3.7\%) use nested \texttt{SELECT} queries, and touch as many as 7 separate tables (in 616 cases).  
 This suggests extensive use of \texttt{DELETE} as a form of garbage-collection or cache invalidation, where the invalidation policy is expressed through SQL.
 
 %\ask{Suggestion is one thing... how might we validate the claim that DELETE is used for cache invalidation?}
@@ -208,13 +209,13 @@ Slightly over 1 million statements executed by SQLite over the course of the mon
 
 193 of the \texttt{UPDATE} statements relied on a nested \texttt{SELECT} statement as part of their \texttt{WHERE} clause, including 56 that involved 2 levels of nesting.  Of the 193 \texttt{UPDATE}s with nested subqueries, 25 also involved aggregation.  
 
+
 Although the \texttt{WHERE} clause of the updates included a variety of expressions, \textit{every single setter} in every \texttt{UPDATE} statement in the trace assigned a constant value;
 % , as in the following statement:
 % \begin{verbatim}
 % UPDATE ScheduledTaskProto SET value=?,key=?,sortingValue=? WHERE key = ?;
 % \end{verbatim}
-Not a single \texttt{UPDATE} expression attempted to compute new values using SQL, suggesting a strong preference for computing updated values in the application itself.  This is not entirely unexpected, as the database lives in the address space of the application.  Consequently, it is feasible to first perform a \texttt{SELECT} to read values out of the database and then perform an \texttt{UPDATE} to write out the changes, a tactic used by many ORMs.  An unfortunate consequence of this tactic is that ORMs cache database objects at the application layer unnecessarily, suggesting that a stronger coupling between SQL and Java (\textit{e.g.}, through language primitives like LINQ~\cite{box2007linq} or StatusQuo~\cite{cheung2013statusquo}% or Truffle~\cite{wimmer2012truffle}) 
-could be of significant benefit for Android developers.
+Not a single \texttt{UPDATE} expression attempted to compute new values using SQL, suggesting a strong preference for computing updated values in the application itself.  This is not entirely unexpected, as the database lives in the address space of the application.  Consequently, it is feasible to first perform a \texttt{SELECT} to read values out of the database and then perform an \texttt{UPDATE} to write out the changes, a tactic used by many ORMs.  An unfortunate consequence of this tactic is that ORMs cache database objects at the application layer unnecessarily, suggesting that a stronger coupling between SQL and Java (\textit{e.g.}, through language primitives like LINQ~\cite{box2007linq} or StatusQuo~\cite{cheung2013statusquo}) could be of significant benefit to Android developers.
 
 \begin{figure}
 \centering
@@ -240,14 +241,15 @@ could be of significant benefit for Android developers.
 \subsubsection{Per-Application Analysis}
 
 \begin{figure}
-\centering
   \begin{subfigure}[t]{0.5\textwidth}
-	\includegraphics[width=\textwidth]{graphs/data_mod_ops_cdf_by_app}
+    \centering
+	\includegraphics[width=0.9\textwidth]{graphs/data_mod_ops_cdf_by_app}
     \caption{}
     \label{fig:updateByApp:modOps}
   \end{subfigure}%
   \begin{subfigure}[t]{0.5\textwidth}
-	\includegraphics[width=\textwidth]{graphs/read_write_ratio_cdf_by_app}
+    \centering
+	\includegraphics[width=0.9\textwidth]{graphs/read_write_ratio_cdf_by_app}
     \caption{}
     \label{fig:updateByApp:writeRatio}
   \end{subfigure}%

sections/5-dba.tex

@@ -84,7 +84,7 @@ applications seem to be using the SQLite database almost as a key-value store.
 
 \subsubsection{Runtime Characteristics by Query Type}
 
-Figure~\ref{fig-type} show runtime characteristics for each of 
+Figure~\ref{fig-type} shows runtime characteristics for each of 
 the four types of SQL statement.  Figure~\ref{fig-type-interarrival-prev} and 
 \ref{fig-type-interarrival-next} in particular show the time since the last
 query to be issued and the time until the next query is issued (respectively), 
@@ -154,7 +154,7 @@ Figure~\ref{fig-app-rowcount} shows that the \textit{Android System} and
 The remaining apps issue a large number of single-row queries --- Even 
 \textit{Contacts Storage} has a workload consisting of 45\% single-row reads --- 
 the number of rows returned in general varies much more widely.  Many of these 
-apps' user interfaces have both a list and search views that shows multiple records
+apps' user interfaces have both a list and a search view that show multiple records
 at a time, suggesting that these views are backed directly by SQLite.  Although all
 apps have long tails, two apps in particular: \textit{Gmail} and \textit{Google+} are
 notable for regularly issuing queries that return on the order of 100 rows.  

sections/6-pocketdata.tex

@@ -3,7 +3,7 @@ In spite of the prevalence of mobile devices, relatively little attention has be
 \subsection{Pocket Data Management}
 \label{sec:pocketdata:related}
 
-Kang et. al.~\cite{kang2013xftl} explored the design of a flash-aware transactional layer called X-FTL, specifically targeting limitations of SQLite's undo/redo logging on mobile devices.  To evaluate their work, the authors used the TPC-C benchmark in conjunction with a series of micro-benchmarks that evaluate the file system's response to database write operations.  This workload is appropriate for their target optimizations.  However, as we discuss below, TPC-C is not sufficiently representative of a pocket data workload to be used as a general-purpose mobile database benchmark.
+Kang et. al.~\cite{kang2013xftl} explored the design of a flash-aware transactional layer called X-FTL, specifically targeting limitations of SQLite's redo logging on mobile devices.  To evaluate their work, the authors used the TPC-C benchmark in conjunction with a series of micro-benchmarks that evaluate the file system's response to database write operations.  This workload is appropriate for their target optimizations.  However, as we discuss below, TPC-C is not sufficiently representative of a pocket data workload to be used as a general-purpose mobile database benchmark.
 
 Jeong et. al.~\cite{jeong2013iostack} noted similar limitations in SQLite's transactional layer, and went about streamlining the IO-stack, again primarily for the benefit of mobile devices.  Again, micro-benchmarks played a significant role in the author's evaluation of their work.  To evaluate their system's behavior under real-world conditions, the authors ran the \textit{Twitter} and \textit{Facebook} apps, simulating user behavior using a mobility trace generated by MobiGen~\cite{ahmed2009mobigen}.  This is perhaps the most representative benchmarking workload that we encountered in our survey of related work.  %However, it too could be improved.
 %In our traces, Facebook and Twitter do represent a substantial contribution to the database workload of a typical smartphone, but still perform orders of magnitude less work with SQLite than built-in apps and system services.
@@ -18,11 +18,12 @@ Given the plethora of available benchmarking software, it is reasonable to ask w
 
 Although no explicit macro-benchmarks exist for mobile embedded databases, we note two benchmark data generators that do simulate several properties of interest: AndroBench~\cite{kim2012androbench} and MobiGen~\cite{ahmed2009mobigen}.  AndroBench is a micro-benchmark capable of simulating the IO behavior of SQLite under different workloads.  It is primarily designed to evaluate the file-system supporting SQLite, rather than the embedded database itself.  However, the structure of its micro-benchmark workloads can just as effectively be used to compare two embedded database implementations.
 
-The second benchmark, MobiGen has little to do with data management directly.  Rather, it generates realistic traces of environmental inputs (\textit{e.g.}, signal strength, accelerometer readings, \textit{etc}\ldots), simulating the effects of a phone being carried through a physical space.  Replaying these traces through a virtual machine running a realistic application workload could generate realistic conditions (\textit{e.g.}, as in the evaluation of X-FTL~\cite{jeong2013iostack}).  However, it does not simulate the effects of user interactions with apps running on the device.
+The second benchmark, MobiGen has little to do with data management directly.  Rather, it generates realistic traces of environmental inputs, %(\textit{e.g.}, signal strength, accelerometer readings, \textit{etc}\ldots), 
+simulating the effects of a phone being carried through a physical space.  Replaying these traces through a virtual machine running a realistic application workload could generate realistic conditions (\textit{e.g.}, as in the evaluation of X-FTL~\cite{jeong2013iostack}).  However, it can not simulate the effects of user interactions with apps running on the device.
 
 \subsubsection{TPC-C} 
 
-One macro-benchmark suite that bears a close resemblance to the trace workload is TPC-C~\cite{tpcc}, which simulates a supply-chain management system.  It includes a variety of transactional tasks ranging from low-latency user interactions for placing and querying orders, to longer-running batch processes that simulate order fulfillment.  A key feature of this benchmark workload is the level of concurrency expected and required of the system.  Much of the data is neatly partitioned, but the workload is designed to force a non-trivial level of cross-talk between partitions, making concurrency a bottleneck at higher throughput.  Conversely, mobile SQLite databases are isolated into specialized app-specific silos.  In our experiments, throughput remained at very manageable levels from a concurrency standpoint.  The most intensive database user, \textit{Google Play services} had 14.8 million statements attributable to it, just under half of which were writes.  This equates to about one write every 3 seconds, which is substantial from a power management and latency perspective, but not from the standpoint of concurrency.
+One macro-benchmark suite that bears a close resemblance to the trace workload is TPC-C~\cite{tpcc}, which simulates a supply-chain management system.  It includes a variety of transactional tasks ranging from low-latency user interactions for placing and querying orders, to longer-running batch processes that simulate order fulfillment.  A key feature of this benchmark workload is the level of concurrency expected and required of the system.  Much of the data is neatly partitioned, but the workload is designed to force a non-trivial level of cross-talk between partitions, making concurrency a bottleneck at higher throughputs.  Conversely, mobile SQLite databases are isolated into specialized app-specific silos.  In our experiments, throughput remained at very manageable levels from a concurrency standpoint.  The most intensive database user, \textit{Google Play services} had 14.8 million statements attributable to it, just under half of which were writes.  This equates to about one write every 3 seconds, which is substantial from a power management and latency perspective, but not from the standpoint of concurrency.
 
 \subsubsection{YCSB}
 
@@ -31,7 +32,7 @@ The Yahoo Cloud Services benchmark~\cite{ycsb} is designed to capture a variety
 
 \subsubsection{Analytics}
 
-These more complex queries include multiple levels of query nesting, wide joins, and extensive use of aggregation.  As such, they more closely resemble analytics workload benchmarks such as TPC-H~\cite{tpch}, The Star-Schema Benchmark~\cite{ssb}, and TPC-DS~\cite{tpcds}.  This resemblance is more than passing; many of the more complex queries we encountered appeared to be preparing application run-time state for presentation to the user.  For example the \textit{Google Play Games} service tracks so-called \textit{events} and \textit{quests}, and participating \textit{apps}.  One of the most complex queries that we encountered appeared to be linking and summarizing these features together for presentation in a list view.  Additionally, we note that the presence of analytics queries in pocket data management workloads is likely to increase further, as interest grows in smart phones as a platform for personal sensing~\cite{campbell2008peoplesensing,klasnja2009using,lam2009healthmonitoring}.
+These more complex queries include multiple levels of query nesting, wide joins, and extensive use of aggregation.  As such, they more closely resemble analytics workload benchmarks such as TPC-H~\cite{tpch}, The Star-Schema Benchmark~\cite{ssb}, and TPC-DS~\cite{tpcds}.  This resemblance is more than passing; many of the more complex queries we encountered appeared to be preparing application run-time state for presentation to the user.  For example the \textit{Google Play Games} service tracks so-called \textit{events} and \textit{quests}, and participating \textit{apps}.  One of the most complex queries that we encountered appeared to be linking and summarizing these features together for presentation in a list view.  We note that the presence of analytics queries in pocket data management is likely to increase further, as interest grows in smart phones as a platform for personal sensing~\cite{campbell2008peoplesensing,klasnja2009using,lam2009healthmonitoring}.
 
 \subsubsection{TPC-E}
 

sections/7-conclusions.tex

@@ -0,0 +1 @@
+In this paper, we identified embedded databases on smartphones as the foundation of a new class of \textit{pocket data} workloads.  We have presented the preliminary results for a long-running study of SQLite embedded database usage on Android smartphones, and identified numerous ways in which pocket data workloads differ from big data workloads.  Through this study, we hope to be able to create a benchmark that will spur further research and development on pocket data and embedded databases.