Tyson
1731f1fed4
Added an optional format parameter for individual job queries and optimized the jobId query
2013-01-11 15:01:43 -05:00
Tyson
c063e8777e
Added implicit json writers for JobDescription and ExecutorRunner
2013-01-11 14:57:38 -05:00
Matei Zaharia
2e914d9983
Formatting
2013-01-10 19:13:08 -08:00
Matei Zaharia
3548c9c0c8
Merge branch 'master' of github.com:mesos/spark
2013-01-10 19:06:40 -08:00
Matei Zaharia
6d1c230281
Merge pull request #357 from tysonjh/master
...
JSON support added to WebUI
2013-01-10 19:06:07 -08:00
Matei Zaharia
248995c535
Merge pull request #356 from shane-huang/master
...
Fix an issue in ConnectionManager where sendMessage may create too many unnecessary connections
2013-01-10 17:52:23 -08:00
shane-huang
9930a95d21
Modified Patch according to comments
2013-01-10 20:09:55 +08:00
Tyson
549ee388a1
Removed io.spray spray-json dependency as it is not needed.
2013-01-09 15:12:23 -05:00
Tyson
bf9d9946f9
Query parameter reformatted to be more extensible and routing more robust
2013-01-09 11:29:58 -05:00
Tyson
0da2ff102e
Added url query parameter json and handler
2013-01-09 10:40:48 -05:00
Tyson
269fe018c7
JSON object definitions
2013-01-09 10:40:43 -05:00
Matei Zaharia
9cc764f523
Code style
2013-01-08 22:29:57 -08:00
Matei Zaharia
14972141f9
Merge pull request #344 from mbautin/log_preferred_hosts
...
Log preferred hosts
2013-01-08 22:26:34 -08:00
Stephen Haberman
8ac0f35be4
Add JavaRDDLike.keyBy.
2013-01-08 09:57:45 -06:00
Stephen Haberman
4ee6b22775
Merge branch 'master' into tupleBy
...
Conflicts:
core/src/test/scala/spark/RDDSuite.scala
2013-01-08 09:10:10 -06:00
shane-huang
e4cb72da8a
Fix an issue in ConnectionManager where sendingMessage may create too many unnecessary SendingConnections.
2013-01-08 22:40:58 +08:00
Mikhail Bautin
4725b0f643
Fixing if/else coding style for preferred hosts logging
2013-01-07 20:09:26 -08:00
Mikhail Bautin
c41042c816
Log preferred hosts
2013-01-07 20:06:09 -08:00
Shivaram Venkataraman
77d751731c
Remove unused BoundedMemoryCache file and associated test case.
2013-01-07 15:57:46 -08:00
Shivaram Venkataraman
aed368a970
Update Hadoop dependency to 1.0.3 as 0.20 has Sun specific dependencies. Also
...
fix SequenceFileRDDFunctions to pick the right type conversion across Hadoop
versions
2013-01-07 15:57:33 -08:00
Shivaram Venkataraman
f8d579a0c0
Remove dependencies on sun jvm classes. Instead use reflection to infer
...
HotSpot options and total physical memory size
2013-01-07 15:57:18 -08:00
Matei Zaharia
1941d9602d
Merge branch 'master' of github.com:mesos/spark
2013-01-07 16:50:39 -05:00
Matei Zaharia
9c32f300fb
Add Accumulable.setValue for easier use in Java
2013-01-07 16:50:23 -05:00
Stephen Haberman
8dc06069fe
Rename RDD.tupleBy to keyBy.
2013-01-06 15:21:45 -06:00
Matei Zaharia
8fd3a70c18
Add PairRDD.keys() and values() to Java API
2013-01-05 22:46:45 -05:00
Matei Zaharia
b1663752c6
Merge pull request #351 from stephenh/values
...
Add PairRDDFunctions.keys and values.
2013-01-05 19:15:54 -08:00
Matei Zaharia
0982572519
Add methods called just 'accumulator' for int/double in Java API
2013-01-05 22:11:28 -05:00
Matei Zaharia
86af64b0a6
Fix Accumulators in Java, and add a test for them
2013-01-05 20:55:17 -05:00
Stephen Haberman
1fdb6946b5
Add RDD.tupleBy.
2013-01-05 13:07:59 -06:00
Stephen Haberman
f4e6b9361f
Add RDD.collect(PartialFunction).
2013-01-05 12:14:08 -06:00
Stephen Haberman
8d57c78c83
Add PairRDDFunctions.keys and values.
2013-01-05 12:04:01 -06:00
Matei Zaharia
55809fbc6d
Merge pull request #349 from woggling/cache-finally
...
Avoid stalls when computation of cached RDD throws exception
2013-01-01 08:21:33 -08:00
Charles Reiss
58072a7340
Remove some dead comments
2013-01-01 08:07:44 -08:00
Charles Reiss
feadaf72f4
Mark key as not loading in CacheTracker even when compute() fails
2013-01-01 07:57:20 -08:00
Josh Rosen
f803953998
Raise exception when hashing Java arrays (SPARK-597)
2012-12-31 20:20:11 -08:00
Josh Rosen
397e67103c
Change Utils.fetchFile() warning to SparkException.
2012-12-28 17:37:13 -08:00
Josh Rosen
d64fa72d2e
Add addFile() and addJar() to JavaSparkContext.
2012-12-28 17:00:57 -08:00
Josh Rosen
bd237d4a9d
Add synchronization to LocalScheduler.updateDependencies().
2012-12-28 17:00:57 -08:00
Josh Rosen
f1bf4f0385
Skip deletion of files in clearFiles().
...
This fixes an issue where Spark could delete
original files in the current working directory
that were added to the job using addFile().
There was also the potential for addFile() to
overwrite local files, which is addressed by
changing Utils.fetchFile() to log a warning
instead of overwriting a file with new contents.
This is a short-term fix; a better long-term
solution would be to remove the dependence on
storing files in the current working directory,
since we can't change the cwd from Java.
2012-12-28 17:00:57 -08:00
Mark Hamstra
903f3518df
fall back to filter-map-collect when calling lookup() on an RDD without a partitioner
2012-12-24 13:18:45 -08:00
Mark Hamstra
61be8566e2
Allow distinct() to be called without parentheses when using the default number of splits.
2012-12-24 02:36:47 -08:00
Reynold Xin
60f7338092
Remove the call to close input stream in Kryo serializer.
2012-12-21 15:49:33 -08:00
Matei Zaharia
3334b7c6b5
Merge pull request #341 from rxin/4a3fb06ac2d11125feb08acbbd4df76d1e91b677
...
Kryo2 update against Spark master
2012-12-21 15:31:23 -08:00
Matei Zaharia
5e51b889fe
Merge pull request #327 from rxin/spark-633
...
Added the ability in block manager to remove blocks.
2012-12-20 11:33:38 -08:00
Reynold Xin
9397c5014e
Let the slave notify the master block removal.
2012-12-20 01:37:09 -08:00
Reynold Xin
68c52d80ec
Moved BlockManager's IdGenerator into BlockManager object. Removed some
...
excessive debug messages.
2012-12-19 15:27:23 -08:00
Patrick Wendell
bfac06e1f6
SPARK-616: Logging dead workers in Web UI.
...
This patch keeps track of which workers have died and marks them
as such in the master web UI. It also handles workers which die and
re-register using different actor ID's.
2012-12-17 23:09:05 -08:00
Matei Zaharia
b82a6dd2c7
Merge pull request #332 from JoshRosen/spark-607
...
Add try-finally to handle MapOutputTracker timeouts
2012-12-14 11:41:16 -08:00
Reynold Xin
06f855c24d
Merge branch 'spark-633' of github.com:rxin/spark into spark-633
2012-12-14 00:27:24 -08:00
Reynold Xin
8c01295b85
Fixed conflicts from merging Charles' and TD's block manager changes.
2012-12-14 00:26:36 -08:00