ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
1929 commits 6 branches 181 tags 397 MiB
Commit graph

1929 commits

This branch
This branch
All branches
Author SHA1 Message Date
Tyson 269fe018c7 JSON object definitions 2013-01-09 10:40:43 -05:00
Tyson 6e8c8f61c4 Added the spray implicit marshaller library 
Added the io.spray JSON library
 2013-01-09 10:40:33 -05:00
Matei Zaharia 9cc764f523 Code style 2013-01-08 22:29:57 -08:00
Matei Zaharia 14972141f9 Merge pull request #344 from mbautin/log_preferred_hosts 
Log preferred hosts
 2013-01-08 22:26:34 -08:00
Matei Zaharia d0bae072ea Merge pull request #353 from stephenh/tupleBy 
Add RDD.tupleBy.
 2013-01-08 22:24:03 -08:00
Josh Rosen b57dd0f160 Add mapPartitionsWithSplit() to PySpark. 2013-01-08 16:05:02 -08:00
Stephen Haberman c3f1675f9c Retrieve jars to a flat directory so * can be used for the classpath. 2013-01-08 14:44:33 -06:00
Stephen Haberman 8ac0f35be4 Add JavaRDDLike.keyBy. 2013-01-08 09:57:45 -06:00
Stephen Haberman 4ee6b22775 Merge branch 'master' into tupleBy 
Conflicts:
	core/src/test/scala/spark/RDDSuite.scala
 2013-01-08 09:10:10 -06:00
shane-huang e4cb72da8a Fix an issue in ConnectionManager where sendingMessage may create too many unnecessary SendingConnections. 2013-01-08 22:40:58 +08:00
Shivaram Venkataraman f7adb382ac Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now 
by using -Dhadoop -Phadoop2.
 2013-01-08 03:19:43 -08:00
Patrick Wendell 6c502e3793 Making the Twitter example distributed. 
This adds a distributed (receiver-based) implementation of the
Twitter dstream. It also changes the example to perform a
distributed sort rather than collecting the dataset at one node.
 2013-01-07 22:01:11 -08:00
Mikhail Bautin 4725b0f643 Fixing if/else coding style for preferred hosts logging 2013-01-07 20:09:26 -08:00
Mikhail Bautin c41042c816 Log preferred hosts 2013-01-07 20:06:09 -08:00
Tathagata Das 8c1b872512 Moved Twitter example to the where the other examples are. 2013-01-07 17:48:10 -08:00
Shivaram Venkataraman 4bbe07e5ec Activate hadoop1 profile by default for maven builds 2013-01-07 17:46:22 -08:00
Matei Zaharia f7cf035b9b Merge pull request #350 from tdas/streaming 
Spark Streaming
 2013-01-07 17:40:11 -08:00
Matei Zaharia a37adfa67b Merge pull request #354 from shivaram/ibm-jdk-fixes 
Fixes to build and test spark on IBM JVM
 2013-01-07 17:37:03 -08:00
Shivaram Venkataraman b1336e2fe4 Update expected size of strings to match our dummy string class 2013-01-07 17:00:32 -08:00
Tathagata Das 64dceec293 Merge branch 'streaming-merge' into dev-merge 2013-01-07 16:54:35 -08:00
Shivaram Venkataraman fb3d4d5e85 Make default hadoop version 1.0.3 in pom.xml 2013-01-07 16:46:06 -08:00
Tathagata Das d808e1026a Merge branch 'dev' into dev-merge 2013-01-07 16:41:11 -08:00
Tathagata Das 1d8b1c9bec Merge branch 'dev-merge' of github.com:radlab/spark into dev-merge 2013-01-07 16:14:11 -08:00
Tathagata Das 4719e6d8fe Changed locations for unit test logs. 2013-01-07 16:06:07 -08:00
Shivaram Venkataraman 55c66d365f Use a dummy string class in Size Estimator tests to make it resistant to jdk 
versions
 2013-01-07 15:58:00 -08:00
Shivaram Venkataraman 77d751731c Remove unused BoundedMemoryCache file and associated test case. 2013-01-07 15:57:46 -08:00
Shivaram Venkataraman aed368a970 Update Hadoop dependency to 1.0.3 as 0.20 has Sun specific dependencies. Also 
fix SequenceFileRDDFunctions to pick the right type conversion across Hadoop
versions
 2013-01-07 15:57:33 -08:00
Shivaram Venkataraman f8d579a0c0 Remove dependencies on sun jvm classes. Instead use reflection to infer 
HotSpot options and total physical memory size
 2013-01-07 15:57:18 -08:00
Tathagata Das e60514d79e Fixed bug 2013-01-07 15:16:16 -08:00
Tathagata Das 3b0a3b89ac Added better docs for RDDCheckpointData 2013-01-07 14:55:49 -08:00
Tathagata Das 237bac36e9 Renamed examples and added documentation. 2013-01-07 14:37:21 -08:00
Matei Zaharia 1941d9602d Merge branch 'master' of github.com:mesos/spark 2013-01-07 16:50:39 -05:00
Matei Zaharia 9c32f300fb Add Accumulable.setValue for easier use in Java 2013-01-07 16:50:23 -05:00
Tathagata Das 1346126485 Changed cleanup to clearOldValues for TimeStampedHashMap and TimeStampedHashSet. 2013-01-07 12:11:27 -08:00
Tathagata Das af8738dfb5 Moved Spark Streaming examples to examples sub-project. 2013-01-06 19:31:54 -08:00
Tathagata Das 934ecc829a Removed streaming-env.sh.template 2013-01-06 14:15:07 -08:00
Stephen Haberman 8dc06069fe Rename RDD.tupleBy to keyBy. 2013-01-06 15:21:45 -06:00
Matei Zaharia 8fd3a70c18 Add PairRDD.keys() and values() to Java API 2013-01-05 22:46:45 -05:00
Matei Zaharia b1663752c6 Merge pull request #351 from stephenh/values 
Add PairRDDFunctions.keys and values.
 2013-01-05 19:15:54 -08:00
Matei Zaharia 0982572519 Add methods called just 'accumulator' for int/double in Java API 2013-01-05 22:11:28 -05:00
Matei Zaharia 86af64b0a6 Fix Accumulators in Java, and add a test for them 2013-01-05 20:55:17 -05:00
Matei Zaharia ecf9c08901 Fix Accumulators in Java, and add a test for them 2013-01-05 20:54:08 -05:00
Stephen Haberman 1fdb6946b5 Add RDD.tupleBy. 2013-01-05 13:07:59 -06:00
Stephen Haberman 6a0db3b449 Fix typo. 2013-01-05 12:56:17 -06:00
Matei Zaharia 7ab9f09140 Merge pull request #352 from stephenh/collect 
Add RDD.collect(PartialFunction).
 2013-01-05 10:17:20 -08:00
Stephen Haberman f4e6b9361f Add RDD.collect(PartialFunction). 2013-01-05 12:14:08 -06:00
Stephen Haberman 8d57c78c83 Add PairRDDFunctions.keys and values. 2013-01-05 12:04:01 -06:00
Josh Rosen 33beba3965 Change PySpark RDD.take() to not call iterator(). 2013-01-03 14:52:21 -08:00
Patrick Wendell c438faeac4 Merge pull request #10 from radlab/datahandler-fix 
Several code-quality improvements to DataHandler.
 2013-01-02 17:07:12 -08:00
Patrick Wendell 2ef993d159 BufferingBlockCreator -> NetworkReceiver.BlockGenerator 2013-01-02 14:19:51 -08:00