ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2025 commits 6 branches 181 tags 397 MiB
Commit graph

2025 commits

This branch
This branch
All branches
Author SHA1 Message Date
Josh Rosen 5b6ea9e9a0 Update checkpointing API docs in Python/Java. 2013-01-20 15:31:41 -08:00
Josh Rosen d0ba80dc72 Add checkpointFile() and more tests to PySpark. 2013-01-20 13:59:45 -08:00
Josh Rosen 7ed1bf4b48 Add RDD checkpointing to Python API. 2013-01-20 13:19:19 -08:00
Matei Zaharia fe85a07511 Merge pull request #361 from mesos/streaming 
Merge Streaming into master
 2013-01-20 12:48:15 -08:00
Matei Zaharia 86057ec7c8 Merge branch 'master' into streaming 
Conflicts:
	core/src/main/scala/spark/api/python/PythonRDD.scala
 2013-01-20 12:47:55 -08:00
Josh Rosen 17035db159 Add __repr__ to Accumulator; fix bug in sc.accumulator 2013-01-20 11:58:57 -08:00
Josh Rosen 9f54d7e1f5 Merge pull request #387 from mateiz/python-accumulators 
Add accumulators to PySpark
 2013-01-20 11:00:36 -08:00
Matei Zaharia 2a8c2a6790 Minor formatting fixes 2013-01-20 10:24:53 -08:00
Matei Zaharia 5d473f050e Merge pull request #376 from MLnick/python-als 
Python ALS example
 2013-01-20 10:21:29 -08:00
Matei Zaharia 922c5ec069 Merge pull request #385 from pwendell/ec2-guide-fix 
Clarifying log directory in EC2 guide
 2013-01-20 10:05:38 -08:00
Patrick Wendell 5f74ead636 Changes based on Matei's comment 2013-01-20 08:59:20 -08:00
Tathagata Das 76ff962edc Merge pull request #380 from tdas/streaming 
Merging pySpark to streaming
 2013-01-20 03:54:46 -08:00
Tathagata Das 33bad85bb9 Fixed streaming testsuite bugs 2013-01-20 03:51:11 -08:00
Matei Zaharia ee5a07955c Fix Python guide to say accumulators are available 2013-01-20 02:11:58 -08:00
Matei Zaharia a23ed25f3c Add a class comment to Accumulator 2013-01-20 02:10:25 -08:00
Matei Zaharia 61b6382a35 Launch accumulator tests in run-tests 2013-01-20 01:59:07 -08:00
Matei Zaharia 8e7f098a2c Added accumulators to PySpark 2013-01-20 01:57:44 -08:00
Tathagata Das 4f8fe58b25 Merge branch 'mesos-streaming' into streaming 
Conflicts:
	core/src/main/scala/spark/api/java/JavaRDDLike.scala
	core/src/main/scala/spark/api/java/JavaSparkContext.scala
	core/src/test/scala/spark/JavaAPISuite.java
 2013-01-20 01:13:56 -08:00
Tathagata Das 214345ceac Fixed issue https://spark-project.atlassian.net/browse/STREAMING-29, along with updates to doc comments in SparkContext.checkpoint(). 2013-01-19 23:50:17 -08:00
Patrick Wendell ecdff861f7 Clarifying log directory in EC2 guide 2013-01-19 22:59:35 -08:00
Imran Rashid d98caa0fa0 Merge remote-tracking branch 'dennybritz/blockmanagerUI' into blockmanager_ui 
Conflicts:
	core/src/main/scala/spark/RDD.scala
	core/src/main/scala/spark/storage/BlockManagerMaster.scala
	core/src/main/scala/spark/storage/StorageLevel.scala
 2013-01-18 18:11:26 -08:00
Patrick Wendell 11bbe23140 Merge pull request #369 from pwendell/streaming-java-api 
Java API For Spark Streaming
 2013-01-17 22:39:30 -08:00
Patrick Wendell 12b72b3e73 NetworkWordCount example 2013-01-17 22:37:56 -08:00
Patrick Wendell c46dd2de78 Moving tests to appropriate directory 2013-01-17 21:43:17 -08:00
Patrick Wendell e0165bf714 Adding queueStream and some slight refactoring 2013-01-17 21:25:49 -08:00
Patrick Wendell 6fba7683c2 Small doc fix 2013-01-17 18:46:24 -08:00
Patrick Wendell ee0314c3b3 Merge branch 'streaming' into streaming-java-api 2013-01-17 18:43:00 -08:00
Patrick Wendell 70ba994d6d Import fixup 2013-01-17 18:41:59 -08:00
Patrick Wendell 2261e62ee5 Style cleanup 2013-01-17 18:41:59 -08:00
Patrick Wendell 82b8707c6b Checkpointing in Streaming java API 2013-01-17 18:41:58 -08:00
Patrick Wendell 61b877c688 Adding flatMap 2013-01-17 18:41:58 -08:00
Patrick Wendell d5570c7968 Adding checkpointing to Java API 2013-01-17 18:41:58 -08:00
Patrick Wendell 8e6cbbc6c7 Adding other updateState functions 2013-01-17 18:41:58 -08:00
Patrick Wendell 2a872335c5 Bug fix and test cleanup 2013-01-17 18:41:58 -08:00
Matei Zaharia 54c0f9f185 Fix code that assumed spark.local.dir is only a single directory 2013-01-17 17:40:55 -08:00
Matei Zaharia b534fd363f Merge pull request #382 from fanuo/master 
HttpBroadcast server cache by default in spark.local.dir instead of java.io.tmpdir
 2013-01-17 17:00:25 -08:00
Fernand Pajot 742bc841ad changed HttpBroadcast server cache to be in spark.local.dir instead of java.io.tmpdir 2013-01-17 16:56:11 -08:00
Matei Zaharia 46644e409d Merge branch 'master' of github.com:mesos/spark 2013-01-17 11:17:19 -08:00
Matei Zaharia 892c32a14b Warn users if they run pyspark or spark-shell without compiling Spark 2013-01-17 11:14:47 -08:00
Nick Pentreath a5ba7a9f32 Use only one update function and pass in transpose of ratings matrix where appropriate 2013-01-17 16:21:00 +02:00
Nick Pentreath a512df551f Fixed index error missing first argument 2013-01-17 16:05:27 +02:00
Nick Pentreath 42fbef3c2a Adding default command line args to SparkALS 2013-01-17 15:54:59 +02:00
Matei Zaharia aff1844155 Merge pull request #381 from squito/remove_threadpool 
remove unused thread pool
 2013-01-16 16:46:42 -08:00
Tathagata Das f466ee44bc Merge branch 'master' into streaming 
Conflicts:
	core/src/main/scala/spark/MapOutputTracker.scala
 2013-01-16 12:57:11 -08:00
Imran Rashid eae698f755 remove unused thread pool 2013-01-16 12:21:37 -08:00
Tathagata Das a805ac4a7c Disabled checkpoint for PairwiseRDD (pySpark). 2013-01-16 10:55:26 -08:00
Matei Zaharia 4beb084f64 Merge pull request #374 from woggling/null-mapout 
Generate FetchFailedException even for cached missing map outputs
 2013-01-15 14:22:29 -08:00
Matei Zaharia 7adfedb0d7 Merge pull request #378 from apsaltis/master 
Updated SCALA_VERSION in run2.cmd to match runtime version of Scala
 2013-01-15 14:20:44 -08:00
Tathagata Das cd1521cfdb Merge branch 'master' into streaming 
Conflicts:
	core/src/main/scala/spark/rdd/CoGroupedRDD.scala
	core/src/main/scala/spark/rdd/FilteredRDD.scala
	docs/_layouts/global.html
	docs/index.md
	run
 2013-01-15 12:08:51 -08:00
Charles Reiss 4078623b9f Remove broken attempt to test fetching case. 2013-01-15 12:05:54 -08:00