Cheng Lian 1c690ddafa [SPARK-12933][SQL] Initial implementation of Count-Min sketch ... This PR adds an initial implementation of count min sketch, contained in a new module spark-sketch under `common/sketch`. The implementation is based on the [`CountMinSketch` class in stream-lib][1]. As required by the [design doc][2], spark-sketch should have no external dependency. Two classes, `Murmur3_x86_32` and `Platform` are copied to spark-sketch from spark-unsafe for hashing facilities. They'll also be used in the upcoming bloom filter implementation. The following features will be added in future follow-up PRs: - Serialization support - DataFrame API integration [1]: aac6b4d23a/src/main/java/com/clearspring/analytics/stream/frequency/CountMinSketch.java [2]: https://issues.apache.org/jira/secure/attachment/12782378/BloomFilterandCount-MinSketchinSpark2.0.pdf Author: Cheng Lian <lian@databricks.com> Closes #10851 from liancheng/count-min-sketch.		2016-01-23 00:34:55 -08:00
..
__init__.py	[SPARK-10359] Enumerate dependencies in a file and diff against it for new pull requests	2015-12-30 12:47:42 -08:00
modules.py	[SPARK-12933][SQL] Initial implementation of Count-Min sketch	2016-01-23 00:34:55 -08:00
shellutils.py	[SPARK-7018][BUILD] Refactor dev/run-tests-jenkins into Python	2015-10-18 22:45:27 -07:00