Tathagata Das
|
0426769f89
|
Modified the block dropping code for better performance.
|
2012-07-26 20:53:45 -07:00 |
|
Matei Zaharia
|
1e2df26c33
|
Merge pull request #145 from squito/exp_accum
add Accumulatable, add corresponding docs & tests for accumulators
|
2012-07-26 17:25:45 -07:00 |
|
|
Matei Zaharia
|
5c5aa2ff81
|
Merge pull request #153 from JoshRosen/new-java-api
Java API
|
2012-07-26 17:20:52 -07:00 |
|
Josh Rosen
|
c5e2810dc7
|
Add persist(), splits(), glom(), and mapPartitions() to Java API.
|
2012-07-26 12:46:47 -07:00 |
|
Imran Rashid
|
0384be3467
|
tasks cannot access value of accumulator
|
2012-07-26 12:38:51 -07:00 |
|
|
Josh Rosen
|
bf61c10072
|
Detect non-zero exit status from PipedRDD process.
|
2012-07-26 11:32:59 -07:00 |
|
|
Josh Rosen
|
2a60c998cc
|
Remove StringOps.split() from Java WordCount.
|
2012-07-25 10:13:06 -07:00 |
|
|
Josh Rosen
|
6a78e88237
|
Minor cleanup and optimizations in Java API.
- Add override keywords.
- Cache RDDs and counts in TC example.
- Clean up JavaRDDLike's abstract methods.
|
2012-07-24 09:47:00 -07:00 |
|
Denny
|
4f4a34c025
|
Stlystic changes
Conflicts:
core/src/test/scala/spark/MesosSchedulerSuite.scala
|
2012-07-23 16:32:20 -07:00 |
|
|
Denny
|
866e6949df
|
Always destroy SparkContext in after block for the unit tests.
Conflicts:
core/src/test/scala/spark/ShuffleSuite.scala
|
2012-07-23 16:29:17 -07:00 |
|
|
Matei Zaharia
|
600e99728d
|
Fix a bug where an input path was added to a Hadoop job configuration twice
|
2012-07-23 16:16:19 -07:00 |
|
|
Matei Zaharia
|
da4298135c
|
Merge pull request #152 from dennybritz/fix/testbeforeafter
Always destroy SparkContext in after block for the unit tests.
|
2012-07-23 16:13:24 -07:00 |
|
|
Denny
|
5656dcdfe5
|
Stlystic changes
|
2012-07-23 10:36:52 -07:00 |
|
|
Josh Rosen
|
042dcbde33
|
Add type annotations to Java API methods.
Add missing Scala Map to java.util.Map conversions.
|
2012-07-22 17:35:29 -07:00 |
|
|
Josh Rosen
|
e23938c3be
|
Use mapValues() in JavaPairRDD.cogroupResultToJava().
|
2012-07-22 15:10:01 -07:00 |
|
|
Josh Rosen
|
460da878fc
|
Improve Java API examples
- Replace JavaLR example with JavaHdfsLR example.
- Use anonymous classes in JavaWordCount; add options.
- Remove @Override annotations.
|
2012-07-22 14:40:39 -07:00 |
|
|
Matei Zaharia
|
840e1b21e4
|
Merge branch 'master' of github.com:mesos/spark
|
2012-07-21 21:58:40 -07:00 |
|
|
Matei Zaharia
|
6f44c0db74
|
Fix a bug where an input path was added to a Hadoop job configuration twice
|
2012-07-21 21:58:28 -07:00 |
|
|
Matei Zaharia
|
d1759c0290
|
Merge pull request #149 from dennybritz/serfix
Instantiating custom serializer using user's classpath
|
2012-07-21 21:54:50 -07:00 |
|
|
Matei Zaharia
|
5122f11b05
|
Use full package name in import
|
2012-07-21 21:53:38 -07:00 |
|
|
Josh Rosen
|
01dce3f569
|
Add Java API
Add distinct() method to RDD.
Fix bug in DoubleRDDFunctions.
|
2012-07-18 17:34:29 -07:00 |
|
|
Denny
|
5559608e6f
|
Always destroy SparkContext in after block for the unit tests.
|
2012-07-18 13:09:50 -07:00 |
|
|
Denny
|
e4dbaf653f
|
syntax errors
|
2012-07-18 12:18:00 -07:00 |
|
|
Denny
|
1d98884548
|
Use extended constructor in the examples.
|
2012-07-18 11:46:03 -07:00 |
|
|
Denny
|
2132c541f0
|
Create the ClassLoader before creating a SparkEnv - SparkEnv must use the loader.
|
2012-07-17 14:05:26 -07:00 |
|
|
Denny
|
2b84b50a85
|
Use Context classloader for Serializer class
|
2012-07-17 13:55:23 -07:00 |
|
|
Imran Rashid
|
7f43ba7ffa
|
one more minor cleanup to scaladoc
|
2012-07-16 18:26:48 -07:00 |
|
|
Imran Rashid
|
913d42c6a0
|
fix up scaladoc, naming of type parameters
|
2012-07-16 18:25:15 -07:00 |
|
|
Imran Rashid
|
85940a7d71
|
rename addToAccum to addAccumulator
|
2012-07-16 18:17:13 -07:00 |
|
Mosharaf Chowdhury
|
85cd9979f2
|
Fix for isLocal
|
2012-07-13 01:13:14 -07:00 |
|
|
Mosharaf Chowdhury
|
1c83fd4b66
|
Merged with Upstream dev
|
2012-07-13 01:08:28 -07:00 |
|
|
Mosharaf Chowdhury
|
bb4ee580fa
|
Cleaning BitTorrentBroadcast code...
|
2012-07-13 01:04:01 -07:00 |
|
|
Mosharaf Chowdhury
|
8ccffe21da
|
Cleaned TreeBroadcast
|
2012-07-13 00:54:25 -07:00 |
|
|
Matei Zaharia
|
a33ca6949c
|
Merge branch 'master' of github.com:mesos/spark
|
2012-07-12 18:38:20 -07:00 |
|
|
Matei Zaharia
|
452330efb4
|
Allow null keys in Spark's reduce and group by
|
2012-07-12 18:37:37 -07:00 |
|
|
Matei Zaharia
|
628bb5ca7f
|
Allow null keys in Spark's reduce and group by
|
2012-07-12 18:36:02 -07:00 |
|
|
Matei Zaharia
|
e2a67a8024
|
Fixes to coarse-grained Mesos scheduler in dealing with failed nodes
|
2012-07-12 18:21:52 -07:00 |
|
|
Imran Rashid
|
86024ca74d
|
add some functionality to Vector, delete copy in AccumulatorSuite
|
2012-07-12 13:08:03 -07:00 |
|
|
Imran Rashid
|
42ce879486
|
move Vector class into core and spark.util package
|
2012-07-12 13:08:03 -07:00 |
|
|
Imran Rashid
|
13cc72cfb5
|
Accumulator now inherits from Accumulable, whcih simplifies a bunch of other things (eg., no +:=)
|
2012-07-12 12:40:10 -07:00 |
|
|
Imran Rashid
|
73935629a1
|
improve scaladoc
|
2012-07-12 09:58:06 -07:00 |
|
|
Imran Rashid
|
30480e6dae
|
add Accumulatable, add corresponding docs & tests for accumulators
|
2012-07-12 09:37:42 -07:00 |
|
|
Matei Zaharia
|
be622cf867
|
Formatting
|
2012-07-11 17:31:44 -07:00 |
|
|
Matei Zaharia
|
e8ae77df24
|
Added more methods for loading/saving with new Hadoop API
|
2012-07-11 17:31:33 -07:00 |
|
|
Matei Zaharia
|
4259d37f84
|
Formatting
|
2012-07-10 11:16:34 -07:00 |
|
|
Matei Zaharia
|
6980b67557
|
Added more methods for loading/saving with new Hadoop API
|
2012-07-10 11:11:35 -07:00 |
|
|
Mosharaf Chowdhury
|
34999d97f5
|
Added stop() to the Broadcast subsystem
|
2012-07-10 01:03:47 -07:00 |
|
|
Mosharaf Chowdhury
|
d6a9680604
|
Slightly better check for isLocal
|
2012-07-10 00:16:47 -07:00 |
|
|
Mosharaf Chowdhury
|
701f49e0d9
|
Refactoring
|
2012-07-09 22:39:47 -07:00 |
|
|
Mosharaf Chowdhury
|
cf1c60a1de
|
Refactoring
|
2012-07-09 22:07:46 -07:00 |
|