Tyson
0da2ff102e
Added url query parameter json and handler
2013-01-09 10:40:48 -05:00
Tyson
269fe018c7
JSON object definitions
2013-01-09 10:40:43 -05:00
Tyson
6e8c8f61c4
Added the spray implicit marshaller library
...
Added the io.spray JSON library
2013-01-09 10:40:33 -05:00
Matei Zaharia
9cc764f523
Code style
2013-01-08 22:29:57 -08:00
Matei Zaharia
14972141f9
Merge pull request #344 from mbautin/log_preferred_hosts
...
Log preferred hosts
2013-01-08 22:26:34 -08:00
Matei Zaharia
d0bae072ea
Merge pull request #353 from stephenh/tupleBy
...
Add RDD.tupleBy.
2013-01-08 22:24:03 -08:00
Josh Rosen
b57dd0f160
Add mapPartitionsWithSplit() to PySpark.
2013-01-08 16:05:02 -08:00
Stephen Haberman
c3f1675f9c
Retrieve jars to a flat directory so * can be used for the classpath.
2013-01-08 14:44:33 -06:00
Stephen Haberman
8ac0f35be4
Add JavaRDDLike.keyBy.
2013-01-08 09:57:45 -06:00
Stephen Haberman
4ee6b22775
Merge branch 'master' into tupleBy
...
Conflicts:
core/src/test/scala/spark/RDDSuite.scala
2013-01-08 09:10:10 -06:00
shane-huang
e4cb72da8a
Fix an issue in ConnectionManager where sendingMessage may create too many unnecessary SendingConnections.
2013-01-08 22:40:58 +08:00
Shivaram Venkataraman
f7adb382ac
Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now
...
by using -Dhadoop -Phadoop2.
2013-01-08 03:19:43 -08:00
Patrick Wendell
6c502e3793
Making the Twitter example distributed.
...
This adds a distributed (receiver-based) implementation of the
Twitter dstream. It also changes the example to perform a
distributed sort rather than collecting the dataset at one node.
2013-01-07 22:01:11 -08:00
Mikhail Bautin
4725b0f643
Fixing if/else coding style for preferred hosts logging
2013-01-07 20:09:26 -08:00
Mikhail Bautin
c41042c816
Log preferred hosts
2013-01-07 20:06:09 -08:00
Tathagata Das
8c1b872512
Moved Twitter example to the where the other examples are.
2013-01-07 17:48:10 -08:00
Shivaram Venkataraman
4bbe07e5ec
Activate hadoop1 profile by default for maven builds
2013-01-07 17:46:22 -08:00
Matei Zaharia
f7cf035b9b
Merge pull request #350 from tdas/streaming
...
Spark Streaming
2013-01-07 17:40:11 -08:00
Matei Zaharia
a37adfa67b
Merge pull request #354 from shivaram/ibm-jdk-fixes
...
Fixes to build and test spark on IBM JVM
2013-01-07 17:37:03 -08:00
Shivaram Venkataraman
b1336e2fe4
Update expected size of strings to match our dummy string class
2013-01-07 17:00:32 -08:00
Tathagata Das
64dceec293
Merge branch 'streaming-merge' into dev-merge
2013-01-07 16:54:35 -08:00
Shivaram Venkataraman
fb3d4d5e85
Make default hadoop version 1.0.3 in pom.xml
2013-01-07 16:46:06 -08:00
Tathagata Das
d808e1026a
Merge branch 'dev' into dev-merge
2013-01-07 16:41:11 -08:00
Tathagata Das
1d8b1c9bec
Merge branch 'dev-merge' of github.com:radlab/spark into dev-merge
2013-01-07 16:14:11 -08:00
Tathagata Das
4719e6d8fe
Changed locations for unit test logs.
2013-01-07 16:06:07 -08:00
Shivaram Venkataraman
55c66d365f
Use a dummy string class in Size Estimator tests to make it resistant to jdk
...
versions
2013-01-07 15:58:00 -08:00
Shivaram Venkataraman
77d751731c
Remove unused BoundedMemoryCache file and associated test case.
2013-01-07 15:57:46 -08:00
Shivaram Venkataraman
aed368a970
Update Hadoop dependency to 1.0.3 as 0.20 has Sun specific dependencies. Also
...
fix SequenceFileRDDFunctions to pick the right type conversion across Hadoop
versions
2013-01-07 15:57:33 -08:00
Shivaram Venkataraman
f8d579a0c0
Remove dependencies on sun jvm classes. Instead use reflection to infer
...
HotSpot options and total physical memory size
2013-01-07 15:57:18 -08:00
Tathagata Das
e60514d79e
Fixed bug
2013-01-07 15:16:16 -08:00
Tathagata Das
3b0a3b89ac
Added better docs for RDDCheckpointData
2013-01-07 14:55:49 -08:00
Tathagata Das
237bac36e9
Renamed examples and added documentation.
2013-01-07 14:37:21 -08:00
Matei Zaharia
1941d9602d
Merge branch 'master' of github.com:mesos/spark
2013-01-07 16:50:39 -05:00
Matei Zaharia
9c32f300fb
Add Accumulable.setValue for easier use in Java
2013-01-07 16:50:23 -05:00
Tathagata Das
1346126485
Changed cleanup to clearOldValues for TimeStampedHashMap and TimeStampedHashSet.
2013-01-07 12:11:27 -08:00
Tathagata Das
af8738dfb5
Moved Spark Streaming examples to examples sub-project.
2013-01-06 19:31:54 -08:00
Tathagata Das
934ecc829a
Removed streaming-env.sh.template
2013-01-06 14:15:07 -08:00
Stephen Haberman
8dc06069fe
Rename RDD.tupleBy to keyBy.
2013-01-06 15:21:45 -06:00
Matei Zaharia
8fd3a70c18
Add PairRDD.keys() and values() to Java API
2013-01-05 22:46:45 -05:00
Matei Zaharia
b1663752c6
Merge pull request #351 from stephenh/values
...
Add PairRDDFunctions.keys and values.
2013-01-05 19:15:54 -08:00
Matei Zaharia
0982572519
Add methods called just 'accumulator' for int/double in Java API
2013-01-05 22:11:28 -05:00
Matei Zaharia
86af64b0a6
Fix Accumulators in Java, and add a test for them
2013-01-05 20:55:17 -05:00
Matei Zaharia
ecf9c08901
Fix Accumulators in Java, and add a test for them
2013-01-05 20:54:08 -05:00
Stephen Haberman
1fdb6946b5
Add RDD.tupleBy.
2013-01-05 13:07:59 -06:00
Stephen Haberman
6a0db3b449
Fix typo.
2013-01-05 12:56:17 -06:00
Matei Zaharia
7ab9f09140
Merge pull request #352 from stephenh/collect
...
Add RDD.collect(PartialFunction).
2013-01-05 10:17:20 -08:00
Stephen Haberman
f4e6b9361f
Add RDD.collect(PartialFunction).
2013-01-05 12:14:08 -06:00
Stephen Haberman
8d57c78c83
Add PairRDDFunctions.keys and values.
2013-01-05 12:04:01 -06:00
Josh Rosen
33beba3965
Change PySpark RDD.take() to not call iterator().
2013-01-03 14:52:21 -08:00
Patrick Wendell
c438faeac4
Merge pull request #10 from radlab/datahandler-fix
...
Several code-quality improvements to DataHandler.
2013-01-02 17:07:12 -08:00