Website/src/ideas/2014-10-23-OnDemandPartitio...

1.3 KiB

title
On-The-Fly Data Partitioning

Partitioning is useful in many scenarios: Distributed or multi-core computing, balancing data for storage or cache lines, or generalizing sampling processes.  The JITD model can be extended capture both data-partitioning and dynamic data repartitioning for improved multi-core performance.  The resulting datastructure must be safe with respect to concurrent access and modification, and we need new policies to properly take advantage of these capabilities.

What does it take to make JITDs be made safe for concurrent access?

How can JITD accessor functions be modified to take advantage of safe concurrent access?

What kind of policies are required to optimally access JITDs?

The first step would be to make a JITD that is safe for concurrent access and modification.  This would require some serious thought about how data in the JITD is accessed: What kind of mutations/accessors are already safe, and how can we analytically decide what is concurrency-safe, and what isn't.

Longer term efforts would look at how to make concurrent access more efficient.

Scope: The initial step of concurrent JITDs as an MS project for 1-2 students.  Further work might be suitable for an MS thesis or early PhD-level work.