Patrick Wendell
e9eba8c3ce
Use different Hadoop version for YARN artifacts.
...
This uses a seperate Hadoop version for YARN artifact. This means when people link against
spark-yarn, things will resolve correctly.
2013-09-13 15:34:57 -07:00
Patrick Wendell
45ec1cc90e
Add git scm url for publishing
2013-09-12 13:47:31 -07:00
Holden Karau
74f710f6cd
Start of working on SPARK-615
2013-09-11 22:35:58 -07:00
Patrick Wendell
a310de69e0
Merge pull request #926 from kayousterhout/dynamic
...
Changed localProperties to use ThreadLocal (not DynamicVariable).
2013-09-11 19:36:11 -07:00
Mike
d34672f668
Set currentMemory to 0 in clear().
...
Remove unnecessary entries.get() call.
2013-09-11 18:01:19 -07:00
Matei Zaharia
58c7d8b138
Merge pull request #927 from benh/mesos-docs
...
Updated Spark on Mesos documentation.
2013-09-11 17:33:42 -07:00
Benjamin Hindman
8e2602dd70
More updates to Spark on Mesos documentation.
2013-09-11 16:08:54 -07:00
Benjamin Hindman
a0f0c1bed2
Updated Spark on Mesos documentation.
2013-09-11 16:05:25 -07:00
Jey Kottalam
e86d1d4a52
Clarify error messages on SSH failure
2013-09-11 14:59:42 -07:00
Kay Ousterhout
93c4253275
Changed localProperties to use ThreadLocal (not DynamicVariable).
...
The fact that DynamicVariable uses an InheritableThreadLocal
can cause problems where the properties end up being shared
across threads in certain circumstances.
2013-09-11 13:01:39 -07:00
Patrick Wendell
91a59e6b10
Merge pull request #919 from mateiz/jets3t
...
Add explicit jets3t dependency, which is excluded in hadoop-client
2013-09-11 10:21:48 -07:00
Patrick Wendell
b9128d34bf
Merge pull request #922 from pwendell/port-change
...
Change default port number from 3030 to 4030.
2013-09-11 10:03:06 -07:00
Patrick Wendell
bddf135670
Change port from 3030 to 4040
2013-09-11 10:01:38 -07:00
Reynold Xin
e07eef83af
Merge pull request #925 from davidmccauley/master
...
SPARK-894 - Not all WebUI fields delivered VIA JSON
2013-09-11 07:35:39 -07:00
David McCauley
5dd875c5b5
SPARK-894 - Not all WebUI fields delivered VIA JSON
2013-09-11 10:46:37 +01:00
Matei Zaharia
8432f27fbe
Merge pull request #923 from haoyuan/master
...
fix run-example script
2013-09-10 23:19:53 -07:00
Haoyuan Li
45d964b606
better expression
2013-09-10 23:18:22 -07:00
Patrick Wendell
d40f1403f3
Merge pull request #921 from pwendell/master
...
Fix HDFS access bug with assembly build.
2013-09-10 23:05:29 -07:00
Haoyuan Li
56b9407848
fix run-example script
2013-09-10 23:03:09 -07:00
Patrick Wendell
0c1985b153
Fix HDFS access bug with assembly build.
...
Due to this change in HDFS:
https://issues.apache.org/jira/browse/HADOOP-7549
there is a bug when using the new assembly builds. The symptom is that any HDFS access
results in an exception saying "No filesystem for scheme 'hdfs'". This adds a merge
strategy in the assembly build which fixes the problem.
2013-09-10 22:05:13 -07:00
Matei Zaharia
2425eb85ca
Update Python API features
2013-09-10 11:12:59 -07:00
Mike
293c758cc0
Remove MemoryStore$Entry.dropPending, unused as of 42e0a68082.
2013-09-10 00:24:35 -07:00
Matei Zaharia
f117dc6d0d
Add explicit jets3t dependency, which is excluded in hadoop-client
2013-09-10 06:39:25 +00:00
shivaram
8c14f4b722
Merge pull request #917 from pwendell/master
...
Document libgfortran dependency for MLBase
2013-09-09 22:07:58 -07:00
Patrick Wendell
cefee1ed1a
Document fortran dependency for MLBase
2013-09-09 21:45:04 -07:00
Matei Zaharia
c81377b9ed
Merge pull request #915 from ooyala/master
...
Get rid of / improve ugly NPE when Utils.deleteRecursively() fails
2013-09-09 20:16:19 -07:00
Matei Zaharia
61d2a010e1
Merge pull request #916 from mateiz/mkdist-fix
...
Fix copy issue in https://github.com/mesos/spark/pull/899
2013-09-09 18:21:01 -07:00
Evan Chan
fdb8b0eec3
Style fix: put body of if within curly braces
2013-09-09 14:29:32 -07:00
Matei Zaharia
f5a8afa6c3
Fix copy issue in https://github.com/mesos/spark/pull/899
2013-09-09 13:47:56 -07:00
Matei Zaharia
a85758c200
Merge pull request #907 from stephenh/document_coalesce_shuffle
...
Add better docs for coalesce.
2013-09-09 13:45:40 -07:00
Evan Chan
27726079e4
Print out more friendly error if listFiles() fails
...
listFiles() could return null if the I/O fails, and this currently results in an ugly NPE which is hard to diagnose.
2013-09-09 12:58:12 -07:00
Matei Zaharia
084fc36961
Merge pull request #912 from tgravescs/ganglia-pom
...
Add metrics-ganglia to core pom file
2013-09-09 12:01:35 -07:00
Y.CORP.YAHOO.COM\tgraves
2186d93285
Add metrics-ganglia to core pom file
2013-09-09 12:37:33 -05:00
Matei Zaharia
0456384939
Merge pull request #911 from pwendell/ganglia-sink
...
Adding Manen dependency for Ganglia
2013-09-09 09:57:54 -07:00
Stephen Haberman
59003d387d
Use a set since shuffle could change order.
2013-09-09 11:45:03 -05:00
Stephen Haberman
6471bfec73
Reword 'evenly distributed' to 'distributed with a hash partitioner.
2013-09-09 11:44:15 -05:00
Patrick Wendell
528fdbae97
Adding Manen dependency
2013-09-09 09:32:18 -07:00
Matei Zaharia
bf984e2745
Merge pull request #890 from mridulm/master
...
Fix hash bug
2013-09-08 23:50:24 -07:00
Reynold Xin
e9d4f44a7a
Merge pull request #909 from mateiz/exec-id-fix
...
Fix an instance where full standalone mode executor IDs were passed to
2013-09-08 23:36:48 -07:00
Matei Zaharia
2447b1c4e6
Merge pull request #910 from mateiz/ml-doc-tweaks
...
Small tweaks to MLlib docs
2013-09-08 22:27:49 -07:00
Matei Zaharia
7a5c4b647b
Small tweaks to MLlib docs
2013-09-08 21:47:24 -07:00
Matei Zaharia
7d3204b056
Merge pull request #905 from mateiz/docs2
...
Job scheduling and cluster mode docs
2013-09-08 21:39:12 -07:00
Matei Zaharia
f1f83712f4
Merge pull request #896 from atalwalkar/master
...
updated content
2013-09-08 21:26:11 -07:00
Matei Zaharia
b458854977
Fix some review comments
2013-09-08 21:25:49 -07:00
Ameet Talwalkar
81a8bd46ac
respose to PR comments
2013-09-08 19:21:30 -07:00
Ameet Talwalkar
bf280c8b0f
Merge remote-tracking branch 'upstream/master'
2013-09-08 18:41:38 -07:00
Patrick Wendell
f68848d95d
Merge pull request #906 from pwendell/ganglia-sink
...
Clean-up of Metrics Code/Docs and Add Ganglia Sink
2013-09-08 18:32:16 -07:00
Matei Zaharia
f9b7f58de2
Fix an instance where full standalone mode executor IDs were passed to
...
StandaloneSchedulerBackend instead of the smaller IDs used within Spark
(that lack the application name).
This was reported by ClearStory in
https://github.com/clearstorydata/spark/pull/9 .
Also fixed some messages that said slave instead of executor.
2013-09-08 18:27:50 -07:00
Matei Zaharia
170b3869ee
Fix unit test failure due to changed default
2013-09-08 17:51:27 -07:00
Ameet Talwalkar
5ac62dbbd0
updates based on comments to PR
2013-09-08 17:39:08 -07:00
