-
-
- Act 1
- Alice wants to analyze two unaligned time series.
+ Demo
-
- | Time | Reading |
|---|
- | 1575731001 | 0 |
- | 1575731014 | 0 |
- | 1575731030 | 0 |
- | 1575731035 | 0 |
- | ... |
- | 1575731219 | 1 |
- | 1575731229 | 1 |
- | 1575731240 | 1 |
-
-
- | Time | Reading |
|---|
- | 1575731011 | 0 |
- | 1575731020 | 0 |
- | 1575731031 | 0 |
- | 1575731039 | 0 |
- | ... |
- | 1575731218 | 1 |
- | 1575731228 | 1 |
- | 1575731237 | 1 |
-
- Step 1: Line up the readings
-
-
-
- Option 1: Do it right
-
-
-
- 
-
-
- |
- Lots of active research efforts!
- |
-
-
- |
- ... but Alice is trying to to GSD!
- |
-
-
-
-
-
- Alice's Observations
-
-
- - Readings every ~10s
- - Readings are binary
- - Readings are incredibly stable
-
-
-
-
-
- INSERT INTO series_one_buckets
- SELECT CAST(time / 10 AS int) AS bucket,
- FIRST(reading)
- FROM series_one
- GROUP BY bucket;
-
- Interpolate missing values
- Hand tune around the switchover as-needed
-
-
-
- Time taken: < 30 minutes
-
-
-
- 
- FreeSVG.org
-
-
-
- Enter Bob...
-
- Similar analysis...
- ... different data
-
- Can Bob re-use Alice's prep+analytics workflow?
-
-
-
- Maybe?
-
- - Are readings still every ~10s?
- - Is the data still binary?
- - Is the data still (relatively) stable?
-
- ... and even then, some manual effort is needed!
-
-
-
- Bob needs to know Alice's assumptions
(and how to use the workflow)?
-
-
-
-
What is a Caveat?
- A brief digression...
+
+
An assumption tied to a fragment of the dataset.
+
If the assumption is wrong, so is the fragment.
+
Classical Databases
-
- One database $D$
-
- Each query gets one answer $R \leftarrow Q(D)$
-
-
-
- Incomplete Databases
-
- Multiple possible databases $D \in \mathcal D$
- (possible worlds)
- Queries get a set of possible answers $\mathcal R \leftarrow \{\; Q(D) \;|\; D \in \mathcal D\;\}$
-
-
-
- Certain tuples exist in all possible worlds. $$certain(\mathcal R) = \bigcap_{R \in \mathcal R} R$$
- Uncertain tuples exist in at least one,
but not all possible worlds. $$uncertain(\mathcal R) = \bigcup_{R \in \mathcal R} R - certain(\mathcal R)$$
- (not limited to set semantics)
-
-
-
- A caveat is an assumption tied to one or more data elements (cells or rows).
- If the assumption is wrong, so is the element.
-
-
-
- Alice / Bob
-
- - FIRST may not pick the right value for a bucket with 2+ distinct values.
- - Interpolation may not pick the right value for a bucket with 0 values.
-
-
-
-
- Carol / Dave
-
- - The model hyperparameters may not work if the data changes too significantly.
- - New values could indicate new data errors that Carol's ingest script hasn't accounted for.
-
-
-
-
- Eve / Hal
-
- - Replacing a parse error with a NULL might not be what Eve expects.
-
-
-
-
- An element has a caveat → The element is uncertain.
-
- ... and btw, here's why.
-
-
-
-
-
- Caveats
-
-
- - Story Time
- - What is a Caveat?
- - The Vizier Notebook
- - Applying Caveats
- - Propagating Caveats
- - Caveats Beyond SQL
-
-
-
-
-
- SELECT setting_1, setting_2, estimate
- FROM Simulation;
-
-
- We want to indicate that the estimate column is only accurate if (for example) P ≠ NP.
-
-
-
- caveat(value, assumption)
-
- returns value, annotated with assumption.
-
-
-
-
- SELECT setting_1, setting_2,
- caveat(estimate, 'Only correct if P ≠ NP')
- AS estimate
- FROM Simulation;
-
- annotation is just a human-readable string.
-
-
-
- Incomplete Databases
-
- caveat() creates 2 sets of possible worlds:
-
- - The assumption holds: value is correct.
- - The assumption does not hold: value is unknown.
-
-
-
-
-
- Alice / Bob
- Mark multi-valued buckets (key repair).
-
- SELECT bucket,
- CASE WHEN bucket_size > 1 THEN
- caveat(reading, 'Picked between two bucket values.')
- ELSE reading END AS reading
- FROM (
- SELECT CAST(time / 10 AS int) AS bucket,
- FIRST(reading) AS reading
- COUNT(*) AS bucket_size
- FROM sensor
- GROUP BY bucket;
- )
-
- Interpolation is more complex... but similar.
-
-
-
- Carol / Dave
- Mark unexpected values the model wasn't trained on.
SELECT
CASE WHEN race_ethnicity
- IN ('White Non-Hispanic', 'Black Non-Hispanic', /* ... */)
- THEN race_ethnicity
+ NOT IN ('White Non-Hispanic', 'Black Non-Hispanic', /* ... */)
- ELSE caveat(race_ethnicity,
+ THEN caveat(race_ethnicity,
'Unexpected race_ethnicity: ' & race_ethnicity)
+ ELSE race_ethnicity
+
END, /* ... */
FROM R
- This check can be automated.
- Eve / Hal
- SELECT /* ... */,
- CASE WHEN CAST(salary AS float) IS NULL THEN
- caveat(NULL, 'Could not cast [ '&salary&' ] to float.')
-
- ELSE CAST(salary AS float) END AS salary
- FROM raw_csv_data;
+
+
+
+ caveat(race_ethnicity,
+ 'Unexpected race_ethnicity: ' & race_ethnicity)
+
+
+
+
+
-
-
-
-
-
- Has anyone asked about "where" provenance?
- Another brief digression...
-
-
- Value Annotations
-
-
- Provenance in Databases: Why, How, and Where
- James Cheney, Laura Chiticariu and Wang-Chiew Tan
-
-
-
- MONDRIAN: Annotating and Querying Databases through Colors and Blocks.
- Floris Geerts, Anastasios Kementsietsidis, Diego Milano
-
-
- and more...
-
-
-
- Value Annotations
- CREATE VIEW Q AS
- SELECT R.A AS X,
- R.B+R.C AS Y
- FROM R
+
+ CASE WHEN /*...*/
+
+
+ THEN caveat(race_ethnicity,
+ 'Unexpected race_ethnicity: ' & race_ethnicity)
+
+ ELSE race_ethnicity
+
+
+
-
- $$annot(\texttt{Q.X}[i]) \leftarrow annot(\texttt{R.A}[i])$$
-
-
- $$annot(\texttt{Q.Y}[i]) \leftarrow annot(\texttt{R.B}[i]) \cup annot(\texttt{R.C}[i])$$
-
- Value Annotations
-
- CREATE VIEW Q AS
- SELECT R.A AS X,
- SUM(R.B) AS Y
- FROM R
-
-
- $$annot(\texttt{Q.X}[i]) \leftarrow \bigcup_{j\;:\;\texttt{R.A}[j] = Q.A[i]} annot(\texttt{R.A}[j])$$
-
-
- $$annot(\texttt{Q.Y}[i]) \leftarrow \bigcup_{j\;:\;\texttt{R.B}[j] = Q.B[i]} annot(\texttt{R.B}[j])$$
-
- ... not the semantics we want
+
+
+
+  |
+
+ →
+ 
+ →
+ |
+  |
+
+
+ | ↑ |
+ |
+ ↑ |
+
+
+ | Assumptions |
+ → |
+ Assumptions |
+
+
+ freesvg.org
-
- Caveats on $\texttt{R.A}$ also affect $\texttt{Q.B}$.
-
+ Propagation
+ Can twiddling the caveatted value change the output?
+
+ $C \leftarrow (a \times X) + Y$
+ Caveats on $X$ and $Y$ propagate to $C$
- Caveats ≠ Value Annotations
-
-
-
-
- 
- 
- 
- 
+ 
+ 
+
+ 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
- 
+ 
-