diff --git a/slides/talks/2019-Waterloo-JITDs/index.html b/slides/talks/2019-Waterloo-JITDs/index.html index aa5e1213..015fdd43 100644 --- a/slides/talks/2019-Waterloo-JITDs/index.html +++ b/slides/talks/2019-Waterloo-JITDs/index.html @@ -247,8 +247,8 @@
  1. What does it mean for a data structure to be halfway between a Binary Tree and a Linked List?
  2. How would we access and manipulate such a data structure?
    3. -
  3. When and how should a data structure transition?
    4. -
  4. How do we automatically generate bespoke data-structures?
    5. + +
  5. How do we automatically generate such data-structures?
@@ -667,19 +667,20 @@ public class TransitionMode extends Mode {
-

Just-in-Time Data Structures

+

Fluid Data Structures

-

A background thread incrementally optimizes the data structure.

-

Continuous availability while performance improves.

+

Encode COG sentences as functional$^*$ data structures.

+

A background thread incrementally applies rewrites to optimize the data structure.

+

Functional data structure allows continuous availability (while performance improves).

Optimizer Work Loop

    -
  1. Which rewrite to apply?
    2. -
  2. On what to apply it?
    3. +
  3. Decide which rewrite to apply and where to apply it.
    4. +
  4. Compute rewritten subtree.
    5. +
  5. Deploy subtree into the data structure.
-

A priority queue keeps track of available rewrite patterns

@@ -711,18 +712,21 @@ public class TransitionMode extends Mode {

Crack

-

Deqeue: 1x Array

-

Enqueue: 2x Array

+

Consume: 1x Array

+

Produce: 2x Array, 1x BinTree

Sort

-

Deqeue: 1x Array

-

Enqueue: 1x Sorted Array

+

Consume: 1x Array

+

Produce: 1x Sorted Array

-

Option 1: Crack($Array_8(1 \ldots 8)$)

-

Option 2: Sort($Array_8(1 \ldots 8)$)

+

Crack and sort both consume an array.

+
+

Option 1: Crack($Array_8(1 \ldots 8)$)

+

Option 2: Sort($Array_8(1 \ldots 8)$)

+
@@ -735,16 +739,19 @@ public class TransitionMode extends Mode {
-

Option 1: Crack($Array_4(1 \ldots 4)$)

+

Option 1: Crack($Array_4(1 \ldots 4)$)

Option 2: Sort($Array_4(1 \ldots 4)$)

+ +
+

The set of rewrites available is a top-1 tree query.

+

Accelerate with materialized views, delta queries, and priority queues.

+
-

How to safely apply rewrites?

- -

Safe pointer hygene!

+

Applying rewrites is expensive in a functional data structure.

@@ -765,7 +772,7 @@ public class TransitionMode extends Mode { Logical Physical - Add a level of indirection + Fluid DSFunctional DS Handle @@ -779,16 +786,12 @@ public class TransitionMode extends Mode {
-
-

Does it work?

-
- -
- -
+
+

Does it work?

+

Incremental Structure Transitions

    @@ -800,22 +803,21 @@ public class TransitionMode extends Mode {
-

Example Policy

-
    -
  1. Crack arrays until threshold
    2. -
  2. Sort unsorted arrays
    3. -
  3. Merge binary trees
    4. -
+

Setup

+
    +
  • Hand Generated C++ Code
    • +
  • $10^9$ records (8-byte key + 8-byte value = 16GB)
    • +
  • Compared vs STL's std::ordered_map, std::unordered_map, Google's cpp-btree
    • +
-

Setup

-
    -
  • DSL compiler (Scala) generates C++ Code
    • -
  • $10^9$ records (8-byte key + 8-byte value = 16GB)
    • -
  • Compared vs STL's std::ordered_map, std::unordered_map, Google's cpp-btree
    • -
  • Crack threshold $10^6 \rightarrow 10^9$ -
+

Example Policy

+
    +
  1. Crack arrays until threshold $10^6 \rightarrow 10^9$
    2. +
  2. Sort unsorted arrays
    3. +
  3. Merge binary trees
    4. +
@@ -828,6 +830,11 @@ public class TransitionMode extends Mode {
+
+

Concurrency / Handles

+ +
+

Load + One Point Lookup

@@ -839,6 +846,7 @@ public class TransitionMode extends Mode {

Ongoing Work

  • Efficiently selecting optimizer targets with view maintenance
    • +
  • DSL compiler to generate C++ code
  • Synthesizing new structures & rules
  • Modeling access and transformation costs
  • Learning optimal policies