Matei Zaharia
a98996d1fe
Merge pull request #545 from ash211/patch-1
...
Don't use deprecated Application in example
2013-03-29 22:12:15 -07:00
Matei Zaharia
104c69414f
Merge pull request #546 from ash211/patch-2
...
Update tuning.md
2013-03-29 22:11:50 -07:00
Jey Kottalam
bc8ba222ff
Bump development version to 0.8.0
2013-03-28 15:42:01 -07:00
Andrew Ash
e8f3669c63
Update tuning.md
...
Make the example more compilable
2013-03-28 19:17:39 -03:00
Jey Kottalam
b569b3f200
Move streaming test initialization into 'before' blocks
2013-03-28 15:08:41 -07:00
Andrew Ash
4e2c965383
Don't use deprecated Application in example
...
As of 2.9.0 extending from Application is not recommended
http://www.scala-lang.org/api/2.9.3/index.html#scala.Application
2013-03-28 17:47:37 -03:00
kalpit
aa9134f72a
spark instance number must be present in log filename to prevent multiple workers from overriding each other's logs
2013-03-26 17:49:30 -07:00
kalpit
f08db010d3
added SPARK_WORKER_INSTANCES : allows spawning multiple worker instances/processes on every slave machine
2013-03-26 17:49:30 -07:00
kalpit
f0164e5047
upgraded sbt version, sbt plugins and some library dependencies to latest stable version
2013-03-26 17:49:29 -07:00
Matei Zaharia
bc36ee4fbb
Merge pull request #543 from holdenk/master
...
Re-enable deprecation warnings and fix deprecated warning.
2013-03-26 15:05:13 -07:00
Andy Konwinski
6c18131487
Add comment to README that 2.10 not yet supported
2013-03-26 14:28:08 -07:00
Holden Karau
1f5381119f
method first in trait IterableLike is deprecated: use `head' instead
2013-03-24 19:19:40 -07:00
Holden Karau
8456d673e2
Re-enable deprecation warnings since there are only two
2013-03-24 17:30:23 -07:00
Holden Karau
e104a76016
Makes the syntax highlighting on the build file not broken in emacs.
2013-03-24 16:16:05 -07:00
Holden Karau
f5df729b12
Explicitly catch all throwables (warning in 2.10)
2013-03-24 16:15:32 -07:00
Stephen Haberman
dd854d5b9f
Use Boolean in the Java API, and != for assert.
2013-03-23 11:49:45 -05:00
Stephen Haberman
4ca273edc4
Merge branch 'master' into shufflecoalesce
...
Conflicts:
core/src/test/scala/spark/RDDSuite.scala
2013-03-23 11:45:45 -05:00
Matei Zaharia
b8949cab88
Merge pull request #505 from stephenh/volatile
...
Make Executor fields volatile since they're read from the thread pool.
2013-03-23 07:19:34 -07:00
Matei Zaharia
fd53f2fc7b
Merge pull request #510 from markhamstra/WithThing
...
mapWith, flatMapWith and filterWith
2013-03-23 07:13:21 -07:00
Stephen Haberman
00170eb0b9
Fix are/our typo.
2013-03-22 12:59:08 -05:00
Stephen Haberman
1c67c7dfd1
Add a shuffle parameter to coalesce.
...
This is useful for when you want just 1 output file (part-00000) but
still up the upstream RDD to be computed in parallel.
2013-03-22 08:54:44 -05:00
Christoph Grothaus
445f387ef4
Bugfix: WorkerWebUI must respect workDirPath from Worker
2013-03-22 11:08:40 +01:00
Matei Zaharia
4c5efcf600
Merge pull request #532 from andyk/master
...
SPARK-715: Adds instructions for building with Maven to documentation
2013-03-20 19:29:23 -07:00
Matei Zaharia
35588490cb
Merge pull request #538 from rxin/cogroup
...
Added mapSideCombine flag to CoGroupedRDD. Added unit test for CoGroupedRDD.
2013-03-20 19:27:47 -07:00
Andy Konwinski
446b801b3b
Fixing typos pointed out by Matei
2013-03-20 17:30:31 -07:00
Stephen Haberman
4f4215311a
Merge branch 'master' into volatile
2013-03-20 15:37:10 -05:00
Matei Zaharia
ca4d083ec8
Merge pull request #528 from MLnick/java-examples
...
[SPARK-707] Adding Java versions of Pi, LogQuery and K-Means examples
2013-03-20 11:22:36 -07:00
Matei Zaharia
b812e6b7bb
Merge pull request #526 from markhamstra/foldByKey
...
Add foldByKey
2013-03-20 11:21:02 -07:00
Nick Pentreath
5a378a1945
Merge branch 'java-examples' of https://github.com/MLnick/spark into java-examples
2013-03-20 09:57:00 +02:00
Nick Pentreath
52398cc1a3
Java indentation 4 --> 2 spaces
2013-03-20 09:55:42 +02:00
Nick Pentreath
7a443bb7d2
Java indentation 2 --> 4 spaces
2013-03-20 09:49:57 +02:00
Reynold Xin
d48ee7e55e
Merge branch 'master' of github.com:mesos/spark into cogroup
2013-03-20 14:00:28 +08:00
Reynold Xin
00a11304fd
Added mapSideCombine flag to CoGroupedRDD. Added unit test for
...
CoGroupedRDD.
2013-03-20 13:49:51 +08:00
Matei Zaharia
945d1e720e
Merge pull request #536 from sasurfer/master
...
CoalescedRDD for many partitions
2013-03-19 21:59:06 -07:00
Matei Zaharia
1cbbe94ac1
Merge pull request #534 from stephenh/removetrycatch
...
Remove try/catch block that can't be hit.
2013-03-19 21:34:34 -07:00
Matei Zaharia
71e53f8894
Merge pull request #537 from wishbear/configurableInputFormat
...
call setConf from input format if it is Configurable
2013-03-19 21:31:41 -07:00
Nick Pentreath
9fa47a2039
A few cosmetic changes for JavaKMeans
2013-03-19 15:31:03 +02:00
Nick Pentreath
568ddf7330
Adding Java K-Means example
2013-03-19 15:29:22 +02:00
Andrey Kouznetsov
bd167f83b0
call setConf from input format if it is Configurable
2013-03-19 17:15:15 +04:00
Giovanni Delussu
aceae029f7
CoalescedRDD changed to work with a big number of partitions both in the original and the new coalesced RDD.
...
The limitation was in the range that Scala.Int can represent.
2013-03-19 11:25:45 +01:00
Stephen Haberman
fb34967815
Remove try/catch block that can't be hit.
2013-03-18 01:55:50 -05:00
Andy Konwinski
ad7f0452ab
Adds page to docs about building using Maven.
...
Adds links to new instructions in:
* The main Spark project README.md
* The docs nav menu called "More"
* The docs Overview page under the "Building" and "Where to Go from Here" sections
2013-03-17 15:02:40 -07:00
Nick Pentreath
b990caeb80
Changes to more closely match line length limit style
2013-03-17 20:03:27 +02:00
Mark Hamstra
ab33e27cc9
constructorOfA -> constructA in doc comments
2013-03-16 15:29:15 -07:00
Mark Hamstra
9784fc1fcd
fix wayward comma in doc comment
2013-03-16 15:25:02 -07:00
Mark Hamstra
32979b5e7d
whitespace
2013-03-16 13:36:46 -07:00
Mark Hamstra
ca9f81e8fc
refactor foldByKey to use combineByKey
2013-03-16 13:31:01 -07:00
Mark Hamstra
1fb192ef40
Merge branch 'master' of https://github.com/mesos/spark into foldByKey
2013-03-16 12:17:13 -07:00
Mark Hamstra
80fc8c82ed
_With[Matei]
2013-03-16 12:16:29 -07:00
Mark Hamstra
38454c4aed
Merge branch 'master' of https://github.com/mesos/spark into WithThing
2013-03-16 11:54:44 -07:00