Mridul Muralidharan
|
5b85c715c8
|
Revert back to 2.0.2-alpha : 0.23.7 has protocol changes which break against cloudera
|
2013-04-24 02:57:51 +05:30 |
|
|
Mridul Muralidharan
|
8faf5c51c3
|
Patch from Thomas Graves to improve the YARN Client, and move to more production ready hadoop yarn branch
|
2013-04-24 02:31:57 +05:30 |
|
|
Mridul Muralidharan
|
b11058f42c
|
Ensure that maven package adds yarn jars as part of shaded jar for hadoop2-yarn profile
|
2013-04-23 22:48:32 +05:30 |
|
|
Mridul Muralidharan
|
7acab3ab45
|
Fix review comments, add a new api to SparkHadoopUtil to create appropriate Configuration. Modify an example to show how to use SplitInfo
|
2013-04-22 08:01:13 +05:30 |
|
|
Mridul Muralidharan
|
ac2e8e8720
|
Add some basic documentation
|
2013-04-19 00:13:19 +05:30 |
|
|
Mridul Muralidharan
|
5ee2f5c483
|
Cache pattern, add (commented out) alternatives for check* apis
|
2013-04-17 23:13:34 +05:30 |
|
|
Mridul Muralidharan
|
f07961060d
|
Add a small note on spark.tasks.schedule.aggression
|
2013-04-17 23:13:02 +05:30 |
|
|
Mridul Muralidharan
|
5d891534fd
|
Move back to 2.0.2-alpha, since 2.0.3-alpha is not available in cloudera yet. Also, add netty dependency explicitly to prevent resolving to older 2.3x version. Additionally, comment out retrievePattern to ensure correct netty is picked up
|
2013-04-17 05:54:43 +05:30 |
|
|
Mridul Muralidharan
|
46779b4745
|
Move back to 2.0.2-alpha, since 2.0.3-alpha is not available in cloudera yet
|
2013-04-17 05:53:28 +05:30 |
|
|
Mridul Muralidharan
|
02dffd2eb0
|
Ensure all ask/await block for spark.akka.askTimeout - so that it is controllable : instead of arbitrary timeouts spread across codebase. In our tests, we use 30 seconds, though default of 10 is maintained
|
2013-04-17 05:52:57 +05:30 |
|
|
Mridul Muralidharan
|
a402b23bcd
|
Fudge order of classpath - so that our jars take precedence over what is in CLASSPATH variable. Sounds logical, hope there is no issue cos of it
|
2013-04-17 05:52:00 +05:30 |
|
|
Mridul Muralidharan
|
bcdde331c3
|
Move from master to driver
|
2013-04-17 04:12:18 +05:30 |
|
|
Mridul Muralidharan
|
ad80f68eb5
|
remove spurious debug statements
|
2013-04-16 22:15:34 +05:30 |
|
|
Mridul Muralidharan
|
f7969f72ee
|
Fix exception when checkpoint path does not exist (no data in rdd which is being checkpointed for example)
|
2013-04-16 21:51:38 +05:30 |
|
|
Mridul Muralidharan
|
323ab8ff3b
|
Scala does not prevent variable shadowing ! Sick error due to it ...
|
2013-04-16 17:05:10 +05:30 |
|
|
Mridul Muralidharan
|
59c380d69a
|
Fix npe
|
2013-04-16 03:29:38 +05:30 |
|
|
Mridul Muralidharan
|
dd2b64ec97
|
Fix bug with atomic update
|
2013-04-16 03:19:24 +05:30 |
|
|
Mridul Muralidharan
|
5540ab8243
|
Use hostname instead of hostport for executor, fix creation of workdir
|
2013-04-16 02:57:43 +05:30 |
|
|
Mridul Muralidharan
|
eb7e95e833
|
Commit job to persist files
|
2013-04-16 02:56:36 +05:30 |
|
|
Mridul Muralidharan
|
19652a44be
|
Fix issue with FileSuite failing
|
2013-04-15 19:16:36 +05:30 |
|
|
Mridul Muralidharan
|
54b3d45b81
|
Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues
|
2013-04-15 18:26:50 +05:30 |
|
|
Mridul Muralidharan
|
d90d2af103
|
Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues
|
2013-04-15 18:12:11 +05:30 |
|
|
Mridul Muralidharan
|
6798a09df8
|
Add support for building against hadoop2-yarn : adding new maven profile for it
|
2013-04-07 17:47:38 +05:30 |
|
Matei Zaharia
|
9831bc1a09
|
Merge pull request #539 from cgrothaus/fix-webui-workdirpath
Bugfix: WorkerWebUI must respect workDirPath from Worker
|
2013-03-29 22:16:22 -07:00 |
|
|
Matei Zaharia
|
3cc8ab6e29
|
Merge pull request #541 from stephenh/shufflecoalesce
Add a shuffle parameter to coalesce.
|
2013-03-29 22:14:07 -07:00 |
|
|
Matei Zaharia
|
cad507adaf
|
Merge pull request #547 from jey/maven-streaming-tests-initialization-fix
Move streaming test initialization into 'before' blocks
|
2013-03-29 22:13:12 -07:00 |
|
|
Matei Zaharia
|
a98996d1fe
|
Merge pull request #545 from ash211/patch-1
Don't use deprecated Application in example
|
2013-03-29 22:12:15 -07:00 |
|
|
Matei Zaharia
|
104c69414f
|
Merge pull request #546 from ash211/patch-2
Update tuning.md
|
2013-03-29 22:11:50 -07:00 |
|
Andrew Ash
|
e8f3669c63
|
Update tuning.md
Make the example more compilable
|
2013-03-28 19:17:39 -03:00 |
|
Jey Kottalam
|
b569b3f200
|
Move streaming test initialization into 'before' blocks
|
2013-03-28 15:08:41 -07:00 |
|
|
Andrew Ash
|
4e2c965383
|
Don't use deprecated Application in example
As of 2.9.0 extending from Application is not recommended
http://www.scala-lang.org/api/2.9.3/index.html#scala.Application
|
2013-03-28 17:47:37 -03:00 |
|
|
Matei Zaharia
|
bc36ee4fbb
|
Merge pull request #543 from holdenk/master
Re-enable deprecation warnings and fix deprecated warning.
|
2013-03-26 15:05:13 -07:00 |
|
Andy Konwinski
|
6c18131487
|
Add comment to README that 2.10 not yet supported
|
2013-03-26 14:28:08 -07:00 |
|
Holden Karau
|
1f5381119f
|
method first in trait IterableLike is deprecated: use `head' instead
|
2013-03-24 19:19:40 -07:00 |
|
|
Holden Karau
|
8456d673e2
|
Re-enable deprecation warnings since there are only two
|
2013-03-24 17:30:23 -07:00 |
|
|
Holden Karau
|
e104a76016
|
Makes the syntax highlighting on the build file not broken in emacs.
|
2013-03-24 16:16:05 -07:00 |
|
|
Holden Karau
|
f5df729b12
|
Explicitly catch all throwables (warning in 2.10)
|
2013-03-24 16:15:32 -07:00 |
|
Stephen Haberman
|
dd854d5b9f
|
Use Boolean in the Java API, and != for assert.
|
2013-03-23 11:49:45 -05:00 |
|
|
Stephen Haberman
|
4ca273edc4
|
Merge branch 'master' into shufflecoalesce
Conflicts:
core/src/test/scala/spark/RDDSuite.scala
|
2013-03-23 11:45:45 -05:00 |
|
|
Matei Zaharia
|
b8949cab88
|
Merge pull request #505 from stephenh/volatile
Make Executor fields volatile since they're read from the thread pool.
|
2013-03-23 07:19:34 -07:00 |
|
|
Matei Zaharia
|
fd53f2fc7b
|
Merge pull request #510 from markhamstra/WithThing
mapWith, flatMapWith and filterWith
|
2013-03-23 07:13:21 -07:00 |
|
|
Stephen Haberman
|
00170eb0b9
|
Fix are/our typo.
|
2013-03-22 12:59:08 -05:00 |
|
|
Stephen Haberman
|
1c67c7dfd1
|
Add a shuffle parameter to coalesce.
This is useful for when you want just 1 output file (part-00000) but
still up the upstream RDD to be computed in parallel.
|
2013-03-22 08:54:44 -05:00 |
|
Christoph Grothaus
|
445f387ef4
|
Bugfix: WorkerWebUI must respect workDirPath from Worker
|
2013-03-22 11:08:40 +01:00 |
|
|
Matei Zaharia
|
4c5efcf600
|
Merge pull request #532 from andyk/master
SPARK-715: Adds instructions for building with Maven to documentation
|
2013-03-20 19:29:23 -07:00 |
|
|
Matei Zaharia
|
35588490cb
|
Merge pull request #538 from rxin/cogroup
Added mapSideCombine flag to CoGroupedRDD. Added unit test for CoGroupedRDD.
|
2013-03-20 19:27:47 -07:00 |
|
|
Andy Konwinski
|
446b801b3b
|
Fixing typos pointed out by Matei
|
2013-03-20 17:30:31 -07:00 |
|
|
Stephen Haberman
|
4f4215311a
|
Merge branch 'master' into volatile
|
2013-03-20 15:37:10 -05:00 |
|
|
Matei Zaharia
|
ca4d083ec8
|
Merge pull request #528 from MLnick/java-examples
[SPARK-707] Adding Java versions of Pi, LogQuery and K-Means examples
|
2013-03-20 11:22:36 -07:00 |
|
|
Matei Zaharia
|
b812e6b7bb
|
Merge pull request #526 from markhamstra/foldByKey
Add foldByKey
|
2013-03-20 11:21:02 -07:00 |
|