Reynold Xin
f19984dafe
More logging changes (task killing for local cluster doesn't work yet).
2013-09-19 18:14:51 -07:00
Reynold Xin
85a0dffe0f
Made task killing work for standalone cluster schedulers.
2013-09-19 16:41:29 -07:00
Reynold Xin
9f8190c17d
Fixed a bug for zero partition in JobWaiter.
2013-09-18 22:42:35 -07:00
Reynold Xin
9332246bd0
Added a hack to kill all active jobs in SparkContext.
2013-09-18 04:38:24 -07:00
Reynold Xin
bf515688e7
Allow SparkContext.submitJob to submit a job for only a subset of the partitions.
2013-09-18 04:16:18 -07:00
Reynold Xin
37d8f37a8e
Added a submitJob interface that returns a Future of the result.
2013-09-17 21:13:59 -07:00
Reynold Xin
1cb42e6b2d
Properly handle job failure when the job gets killed.
2013-09-16 22:10:45 -07:00
Reynold Xin
cbc48be13b
Initial commit for job killing.
2013-09-16 18:54:06 -07:00
Reynold Xin
3443d3fd43
Merge branch 'master' of github.com:mesos/spark
2013-09-16 13:10:35 -07:00
Patrick Wendell
2aff7989ab
Merge pull request #933 from jey/yarn-typo-fix
...
Fix typo in Maven build docs
2013-09-15 14:05:04 -07:00
Jey Kottalam
ac0dd99394
Fix typo in Maven build docs
2013-09-15 13:29:22 -07:00
Patrick Wendell
dbd2c4fd94
Merge pull request #932 from pwendell/mesos-version
...
Bumping Mesos version to 0.13.0
2013-09-15 13:20:41 -07:00
Patrick Wendell
9fb0b9d77f
Merge pull request #931 from pwendell/yarn-docs
...
Explain yarn.version in Maven build docs
2013-09-15 13:02:53 -07:00
Patrick Wendell
c856860c5b
Bumping Mesos version to 0.13.0
2013-09-15 12:46:26 -07:00
Patrick Wendell
362ea0c051
Explain yarn.version in Maven build docs
2013-09-15 12:40:49 -07:00
Patrick Wendell
c4c1db2dd5
Merge pull request #929 from pwendell/master
...
Use different Hadoop version for YARN artifacts.
2013-09-13 19:52:12 -07:00
Patrick Wendell
e9eba8c3ce
Use different Hadoop version for YARN artifacts.
...
This uses a seperate Hadoop version for YARN artifact. This means when people link against
spark-yarn, things will resolve correctly.
2013-09-13 15:34:57 -07:00
Patrick Wendell
45ec1cc90e
Add git scm url for publishing
2013-09-12 13:47:31 -07:00
Patrick Wendell
a310de69e0
Merge pull request #926 from kayousterhout/dynamic
...
Changed localProperties to use ThreadLocal (not DynamicVariable).
2013-09-11 19:36:11 -07:00
Matei Zaharia
58c7d8b138
Merge pull request #927 from benh/mesos-docs
...
Updated Spark on Mesos documentation.
2013-09-11 17:33:42 -07:00
Benjamin Hindman
8e2602dd70
More updates to Spark on Mesos documentation.
2013-09-11 16:08:54 -07:00
Benjamin Hindman
a0f0c1bed2
Updated Spark on Mesos documentation.
2013-09-11 16:05:25 -07:00
Kay Ousterhout
93c4253275
Changed localProperties to use ThreadLocal (not DynamicVariable).
...
The fact that DynamicVariable uses an InheritableThreadLocal
can cause problems where the properties end up being shared
across threads in certain circumstances.
2013-09-11 13:01:39 -07:00
Patrick Wendell
91a59e6b10
Merge pull request #919 from mateiz/jets3t
...
Add explicit jets3t dependency, which is excluded in hadoop-client
2013-09-11 10:21:48 -07:00
Patrick Wendell
b9128d34bf
Merge pull request #922 from pwendell/port-change
...
Change default port number from 3030 to 4030.
2013-09-11 10:03:06 -07:00
Patrick Wendell
bddf135670
Change port from 3030 to 4040
2013-09-11 10:01:38 -07:00
Reynold Xin
e07eef83af
Merge pull request #925 from davidmccauley/master
...
SPARK-894 - Not all WebUI fields delivered VIA JSON
2013-09-11 07:35:39 -07:00
David McCauley
5dd875c5b5
SPARK-894 - Not all WebUI fields delivered VIA JSON
2013-09-11 10:46:37 +01:00
Matei Zaharia
8432f27fbe
Merge pull request #923 from haoyuan/master
...
fix run-example script
2013-09-10 23:19:53 -07:00
Haoyuan Li
45d964b606
better expression
2013-09-10 23:18:22 -07:00
Patrick Wendell
d40f1403f3
Merge pull request #921 from pwendell/master
...
Fix HDFS access bug with assembly build.
2013-09-10 23:05:29 -07:00
Haoyuan Li
56b9407848
fix run-example script
2013-09-10 23:03:09 -07:00
Patrick Wendell
0c1985b153
Fix HDFS access bug with assembly build.
...
Due to this change in HDFS:
https://issues.apache.org/jira/browse/HADOOP-7549
there is a bug when using the new assembly builds. The symptom is that any HDFS access
results in an exception saying "No filesystem for scheme 'hdfs'". This adds a merge
strategy in the assembly build which fixes the problem.
2013-09-10 22:05:13 -07:00
Matei Zaharia
2425eb85ca
Update Python API features
2013-09-10 11:12:59 -07:00
Matei Zaharia
f117dc6d0d
Add explicit jets3t dependency, which is excluded in hadoop-client
2013-09-10 06:39:25 +00:00
shivaram
8c14f4b722
Merge pull request #917 from pwendell/master
...
Document libgfortran dependency for MLBase
2013-09-09 22:07:58 -07:00
Patrick Wendell
cefee1ed1a
Document fortran dependency for MLBase
2013-09-09 21:45:04 -07:00
Matei Zaharia
c81377b9ed
Merge pull request #915 from ooyala/master
...
Get rid of / improve ugly NPE when Utils.deleteRecursively() fails
2013-09-09 20:16:19 -07:00
Matei Zaharia
61d2a010e1
Merge pull request #916 from mateiz/mkdist-fix
...
Fix copy issue in https://github.com/mesos/spark/pull/899
2013-09-09 18:21:01 -07:00
Evan Chan
fdb8b0eec3
Style fix: put body of if within curly braces
2013-09-09 14:29:32 -07:00
Matei Zaharia
f5a8afa6c3
Fix copy issue in https://github.com/mesos/spark/pull/899
2013-09-09 13:47:56 -07:00
Matei Zaharia
a85758c200
Merge pull request #907 from stephenh/document_coalesce_shuffle
...
Add better docs for coalesce.
2013-09-09 13:45:40 -07:00
Evan Chan
27726079e4
Print out more friendly error if listFiles() fails
...
listFiles() could return null if the I/O fails, and this currently results in an ugly NPE which is hard to diagnose.
2013-09-09 12:58:12 -07:00
Matei Zaharia
084fc36961
Merge pull request #912 from tgravescs/ganglia-pom
...
Add metrics-ganglia to core pom file
2013-09-09 12:01:35 -07:00
Y.CORP.YAHOO.COM\tgraves
2186d93285
Add metrics-ganglia to core pom file
2013-09-09 12:37:33 -05:00
Matei Zaharia
0456384939
Merge pull request #911 from pwendell/ganglia-sink
...
Adding Manen dependency for Ganglia
2013-09-09 09:57:54 -07:00
Stephen Haberman
59003d387d
Use a set since shuffle could change order.
2013-09-09 11:45:03 -05:00
Stephen Haberman
6471bfec73
Reword 'evenly distributed' to 'distributed with a hash partitioner.
2013-09-09 11:44:15 -05:00
Patrick Wendell
528fdbae97
Adding Manen dependency
2013-09-09 09:32:18 -07:00
Matei Zaharia
bf984e2745
Merge pull request #890 from mridulm/master
...
Fix hash bug
2013-09-08 23:50:24 -07:00
