ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Fixing the type of the sentiment happiness value

Browse source 
## What changes were proposed in this pull request?

Added the conversion to int for the 'happiness value' read from the file. Otherwise, later on line 75 the multiplication will multiply a string by a number, yielding values like "-2-2" instead of -4.

## How was this patch tested?

Tested manually.

Author: Yury Liavitski <seconds.before@gmail.com>
Author: Yury Liavitski <yury.liavitski@il111.ice.local>

Closes #11540 from heliocentrist/fix-sentiment-value-type.
This commit is contained in:
Yury Liavitski 2016-03-07 10:54:33 +00:00 committed by Sean Owen
parent 4b13896ebf
commit 03f57a6c2d
1 changed files with 2 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
examples/src/main/scala/org/apache/spark/examples/streaming/TwitterHashTagJoinSentiments.scala
View file

@ -56,8 +56,8 @@ object TwitterHashTagJoinSentiments {
val wordSentimentFilePath = "data/streaming/AFINN-111.txt"
val wordSentiments = ssc.sparkContext.textFile(wordSentimentFilePath).map { line =>
val Array(word, happinessValue) = line.split("\t")
(word, happinessValue)
} cache()
(word, happinessValue.toInt)
}.cache()
// Determine the hash tags with the highest sentiment values by joining the streaming RDD
// with the static RDD inside the transform() method and then multiplying