ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

update the deprecated CountMinSketchMonoid function to TopPctCMS function

Browse source 
http://twitter.github.io/algebird/index.html#com.twitter.algebird.legacy.CountMinSketchMonoid$
The CountMinSketchMonoid has been deprecated since 0.8.1. Newer code should use TopPctCMS.monoid().

![image](https://cloud.githubusercontent.com/assets/1327396/7269619/d8b48b92-e8d5-11e4-8902-087f630e6308.png)

Author: KeheCAI <caikehe@gmail.com>

Closes #5629 from caikehe/master and squashes the following commits:

e8aa06f [KeheCAI] update algebird-core to version 0.9.0 from 0.8.1
5653351 [KeheCAI] change scala code style
4c0dfd1 [KeheCAI] update the deprecated CountMinSketchMonoid function to TopPctCMS function
This commit is contained in:
KeheCAI 2015-04-25 08:42:38 -04:00 committed by Sean Owen
parent 59b7cfc41b
commit cca9905b93
2 changed files with 4 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
examples/pom.xml
View file

@ -245,7 +245,7 @@
<dependency>
<groupId>com.twitter</groupId>
<artifactId>algebird-core_${scala.binary.version}</artifactId>
<version>0.8.1</version>
<version>0.9.0</version>
</dependency>
<dependency>
<groupId>org.scalacheck</groupId>

4
examples/src/main/scala/org/apache/spark/examples/streaming/TwitterAlgebirdCMS.scala
View file

@ -18,6 +18,7 @@
package org.apache.spark.examples.streaming
import com.twitter.algebird._
import com.twitter.algebird.CMSHasherImplicits._
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext._
@ -67,7 +68,8 @@ object TwitterAlgebirdCMS {
val users = stream.map(status => status.getUser.getId)
val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC)
// val cms = new CountMinSketchMonoid(EPS, DELTA, SEED, PERC)
val cms = TopPctCMS.monoid[Long](EPS, DELTA, SEED, PERC)
var globalCMS = cms.zero
val mm = new MapMonoid[Long, Int]()
var globalExact = Map[Long, Int]()