Josh Rosen
cda2b15041
Use ec2-metadata in start-slave.sh.
...
PR #419 applied the same change, but only to start-master.sh,
so some workers were still starting their web UI's using internal
addresses.
This should finally fix SPARK-613.
2013-05-24 13:05:06 -07:00
jerryshao
bd3ea8f2a6
fix CheckpointRDD getPreferredLocations java.io.FileNotFoundException
2013-05-24 14:26:19 +08:00
Matei Zaharia
dbbedfc535
Merge pull request #616 from jey/maven-netty-exclusion
...
Exclude old versions of Netty from Maven-based build
2013-05-23 23:11:06 -07:00
Matei Zaharia
a2b0a7975c
Merge pull request #619 from woggling/adjust-sampling
...
Use ARRAY_SAMPLE_SIZE constant instead of hard-coded 100.0 in SizeEstimator
2013-05-21 18:16:20 -07:00
Charles Reiss
f350f14084
Use ARRAY_SAMPLE_SIZE constant instead of 100.0
2013-05-21 18:11:33 -07:00
Reynold Xin
66dac449fb
Merge pull request #618 from woggling/dead-code-disttest
...
DistributedSuite: remove dead code
2013-05-21 11:41:42 -07:00
Charles Reiss
786c97b87c
DistributedSuite: remove dead test code
2013-05-21 11:35:49 -07:00
Ethan Jewett
3217d486f7
Add hBase dependency to examples POM
2013-05-20 19:41:38 -05:00
Andrew xia
ecd6d75c6a
fix bug of unit tests
2013-05-21 06:49:23 +08:00
Jey Kottalam
e7982c798e
Exclude old versions of Netty from Maven-based build
2013-05-18 21:24:58 -07:00
Reynold Xin
5912cc4967
Merge pull request #610 from JoshRosen/spark-747
...
Throw exception if TaskResult exceeds Akka frame size
2013-05-17 19:58:40 -07:00
Reynold Xin
8d78c5f89f
Changed the logging level from info to warning when addJar(null) is
...
called.
2013-05-17 18:51:35 -07:00
Reynold Xin
0eab7a78b9
Fixed a couple typos and formating problems in the YARN documentation.
2013-05-17 18:05:46 -07:00
Reynold Xin
7760d78b3a
Merge branch 'master' of https://github.com/mridulm/spark
2013-05-17 17:58:36 -07:00
Reynold Xin
6729c2ead8
Merge branch 'master' of github.com:mesos/spark
2013-05-17 17:58:06 -07:00
Andrew xia
3d4672eaa9
Merge branch 'master' into xiajunluan
...
Conflicts:
core/src/main/scala/spark/SparkContext.scala
core/src/main/scala/spark/scheduler/cluster/ClusterScheduler.scala
core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala
2013-05-18 07:28:03 +08:00
Andrew xia
d19753b9c7
expose TaskSetManager type to resourceOffer function in ClusterScheduler
2013-05-18 06:45:19 +08:00
Mridul Muralidharan
da2642bead
Fix example jar name
2013-05-17 06:58:46 +05:30
Reynold Xin
6c27c38451
Merge pull request #615 from rxin/build-fix
...
Maven build fix & two other small changes
2013-05-16 17:33:56 -07:00
Reynold Xin
dc146406ae
Updated Scala version in docs generation ruby script.
2013-05-16 17:07:14 -07:00
Reynold Xin
3b3300383a
Updated Scala version in docs generation ruby script.
2013-05-16 16:51:28 -07:00
Reynold Xin
43644a293f
Only check for repl classes if the user is running the repl. Otherwise,
...
check for core classes in run. This fixed the problem that core tests
depend on whether repl module is compiled or not.
2013-05-16 14:31:38 -07:00
Reynold Xin
61cf176238
Added dependency on netty-all in Maven.
2013-05-16 14:31:26 -07:00
Reynold Xin
f742435f18
Removed the duplicated netty dependency in SBT build file.
2013-05-16 14:31:03 -07:00
Andrew xia
c6e2770bfe
Fix ClusterScheduler bug to avoid allocating tasks to same slave
2013-05-17 05:10:38 +08:00
Mridul Muralidharan
f0881f8d48
Hope this does not turn into a bike shed change
2013-05-17 01:58:50 +05:30
Mridul Muralidharan
f16c781709
Fix documentation to use yarn-standalone as master
2013-05-16 17:50:22 +05:30
Mridul Muralidharan
feddd2530d
Filter out nulls - prevent NPE
2013-05-16 17:49:14 +05:30
Mridul Muralidharan
87540a7b38
Fix running on yarn documentation
2013-05-16 15:27:58 +05:30
Josh Rosen
b8e46b6074
Abort job if result exceeds Akka frame size; add test.
2013-05-16 01:57:57 -07:00
Matei Zaharia
2f576aba8f
Merge pull request #602 from rxin/shufflemerge
...
Manual merge & cleanup of Shane's Shuffle Performance Optimization
2013-05-15 18:06:24 -07:00
Reynold Xin
48c6f46a6f
Merge pull request #612 from ash211/patch-4
...
Docs: Mention spark shell's default for MASTER
2013-05-15 10:47:19 -07:00
Andrew Ash
afcad7b3aa
Docs: Mention spark shell's default for MASTER
2013-05-15 14:45:14 -03:00
Reynold Xin
203d7b7c14
Merge pull request #593 from squito/driver_ui_link
...
Master UI has link to Application UI
2013-05-15 00:47:20 -07:00
Reynold Xin
f3491cb89b
Merge branch 'master' of github.com:mesos/spark into shufflemerge
...
Conflicts:
core/src/main/scala/spark/storage/BlockManager.scala
core/src/test/scala/spark/DistributedSuite.scala
project/SparkBuild.scala
2013-05-15 00:31:52 -07:00
Reynold Xin
f9d40a5848
Added a comment in JdbcRDD for example usage.
2013-05-14 23:29:57 -07:00
Reynold Xin
404f9ff617
Added derby dependency to Maven pom files for the JDBC Java test.
2013-05-14 23:28:34 -07:00
Reynold Xin
81ad2fa331
Merge branch 'jdbc' of github.com:koeninger/spark
...
Conflicts:
project/SparkBuild.scala
2013-05-14 23:12:00 -07:00
Imran Rashid
38d4b97c6d
use threads classloader when deserializing task results; classnotfoundexception includes classloader
2013-05-14 22:32:14 -07:00
Imran Rashid
d7d1da79d3
when akka starts, use akkas default classloader (current thread)
2013-05-14 22:32:09 -07:00
Cody Koeninger
b16c4896f6
add test for JdbcRDD using embedded derby, per rxin suggestion
2013-05-14 23:44:04 -05:00
Matei Zaharia
016ac86830
Merge pull request #601 from rxin/emptyrdd-master
...
EmptyRDD (master branch 0.8)
2013-05-13 21:45:36 -07:00
Matei Zaharia
b9aef263df
Merge branch 'master' of github.com:mesos/spark
2013-05-13 17:40:21 -07:00
Matei Zaharia
4b354e0a08
Merge pull request #589 from mridulm/master
...
Add support for instance local scheduling
2013-05-13 17:39:19 -07:00
Matei Zaharia
5dbc9b2006
Merge pull request #608 from pwendell/SPARK-738
...
SPARK-738: Spark should detect and wrap nonserializable exceptions
2013-05-12 11:03:10 -07:00
Patrick Wendell
7f0833647b
Capturing class name
2013-05-12 07:54:03 -07:00
Patrick Wendell
72b9c4cb6e
Small fix
2013-05-11 23:53:50 -07:00
Patrick Wendell
1c15b85051
Removing import
2013-05-11 23:52:53 -07:00
Patrick Wendell
059ab88754
Changing technique to use same code path in all cases
2013-05-11 23:50:54 -07:00
Cody Koeninger
3da2305ed0
code cleanup per rxin comments
2013-05-11 23:59:07 -05:00