Reynold Xin
56774b176e
Added unit test for compression codecs.
2013-07-30 17:12:33 -07:00
Reynold Xin
ad7e9d0d64
CompressionCodec cleanup. Moved it to spark.io package.
2013-07-30 17:11:54 -07:00
Reynold Xin
cb366774c8
Merge pull request #738 from harsha2010/pruning
...
Fix bug in Partition Pruning.
2013-07-26 16:59:30 -07:00
harshars
392d7474fd
Code review
2013-07-26 15:23:15 -07:00
harshars
822aac8f5a
Indentation
2013-07-26 15:10:32 -07:00
harshars
743fc4e7aa
Fix Bug in Partition Pruning, index of Pruned Partitions should inherit from parent
2013-07-26 14:35:17 -07:00
ryanlecompte
fc4b025314
add test
2013-07-24 20:53:15 -07:00
ryanlecompte
a1c515fb02
add copyright back in
2013-07-24 20:50:32 -07:00
ryanlecompte
8e0939f5a9
refactor Kryo serializer support to use chill/chill-java
2013-07-24 20:43:57 -07:00
jerryshao
31ec72b243
Code refactor according to comments
2013-07-24 14:57:47 +08:00
Andrew xia
05637de842
Change class xxxInstrumentation to class xxxSource
2013-07-24 14:57:47 +08:00
Andrew xia
ed1a3bc206
continue to refactor code style and functions
2013-07-24 14:57:47 +08:00
Andrew xia
9cea0c2818
Refactor metricsSystem unit test, add resource files.
2013-07-24 14:57:47 +08:00
jerryshao
e080588f73
Add metrics system unit test
2013-07-24 14:57:47 +08:00
Matei Zaharia
b011329040
Merge pull request #727 from rxin/scheduler
...
Scheduler code style cleanup.
2013-07-23 22:50:09 -07:00
Reynold Xin
3dae1df66f
Moved non-serializable closure catching exception from submitStage to submitMissingTasks
2013-07-23 20:29:07 -07:00
Reynold Xin
85ab8114bc
Moved non-serializable closure catching exception from submitStage to submitMissingTasks
2013-07-23 20:25:58 -07:00
Reynold Xin
f2422d4f29
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure.
2013-07-23 15:30:20 -07:00
Reynold Xin
5ed38b4d1d
Scheduler code style cleanup.
2013-07-23 15:28:59 -07:00
Reynold Xin
101b8cc78a
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure.
2013-07-23 15:28:20 -07:00
Matei Zaharia
f369e0e51b
Merge pull request #720 from ooyala/2013-07/persistent-rdds-api
...
Add a public method getCachedRdds to SparkContext
2013-07-23 13:22:27 -07:00
Evan Chan
efd6418c1b
Move getPersistentRDDs testing to a new Suite
2013-07-23 10:40:41 -07:00
Matei Zaharia
ea1cfabfdd
Merge branch 'master' of github.com:mesos/spark
2013-07-22 16:22:02 -07:00
Matei Zaharia
8e38e77232
Fix a test that was using an outdated config setting
2013-07-22 16:05:32 -07:00
Josh Rosen
f649dabb4a
Fix bug: DoubleRDDFunctions.sampleStdev() computed non-sample stdev().
...
Update JavaDoubleRDD to add new methods and docs.
Fixes SPARK-825.
2013-07-22 13:21:48 -07:00
Evan Chan
0337d88321
Add a public method getCachedRdds to SparkContext
2013-07-21 18:26:14 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Matei Zaharia
b1f9f64743
Merge branch 'master' of github.com:mesos/spark
2013-07-16 11:01:53 -07:00
Matei Zaharia
5c388808a8
SPARK-814: Result stages should be named after action
2013-07-16 11:01:14 -07:00
Matei Zaharia
f347cc3f65
Fix deprecation warning and style issues
2013-07-16 10:53:30 -07:00
Reynold Xin
69316603d6
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 22:50:11 -07:00
Matei Zaharia
5a7835c152
Merge pull request #691 from karenfeng/logpaging
...
Create log pages
2013-07-12 20:28:21 -07:00
Karen Feng
73984b96a8
Removed unit test of nonexistent function Utils.lastNBytes
2013-07-12 14:26:56 -07:00
seanm
a2c915fba8
giving order to top and making tests more clear
2013-07-11 18:55:00 -07:00
Karen Feng
f5f3b272f8
Fixed mixup of start/end, moved more import files
2013-07-10 14:52:29 -07:00
Karen Feng
dbe948d9a2
Moved appropriate import files from UISuite to UtilsSuite
2013-07-10 14:15:41 -07:00
Karen Feng
5f8a20b4a8
Moved unit tests for Utils from UISuite to UtilsSuite
2013-07-10 13:53:39 -07:00
Karen Feng
04263e4d46
Made some minor style changes
2013-07-10 13:15:42 -07:00
Karen Feng
cfb6447ac4
Fixed for nonexistent bytes, added unit tests, changed stdout-page to stdout
2013-07-10 11:47:57 -07:00
seanm
24705d0f46
adding takeOrdered() to RDD
2013-07-10 10:33:11 -07:00
Shivaram Venkataraman
d362d0f411
Ignore stderr when calling cat on a non-existing file
2013-07-07 04:09:46 -07:00
Shivaram Venkataraman
7d6d9e6ab2
Set DriverSuite log level to WARN
2013-07-07 04:09:15 -07:00
Shivaram Venkataraman
a948f06725
Suppress log messages in sbt test with two changes:
...
1. Set akka log level to ERROR before shutting down the actorSystem.
This avoids akka log messages (like Spray) from falling back to INFO
on the Stdout logger
2. Initialize netty to use SLF4J in LocalSparkContext. This ensures that
stack trace thrown during shutdown is handled by SLF4J instead of stdout
2013-07-07 04:09:08 -07:00
Matei Zaharia
1ffadb2d9e
Merge remote-tracking branch 'pwendell/ui-updates'
...
Conflicts:
core/src/main/scala/spark/scheduler/DAGScheduler.scala
core/src/main/scala/spark/util/AkkaUtils.scala
pom.xml
2013-07-06 15:51:41 -07:00
Matei Zaharia
2a36e5449b
Merge pull request #673 from xiajunluan/master
...
Add config template file for fair scheduler feature
2013-07-06 12:43:21 -07:00
Matei Zaharia
399bd65ef5
Fixed compile error due to merge
2013-07-05 11:27:06 -07:00
Matei Zaharia
652ea0f1d8
Allow RDD.takeSample to give samples bigger than the RDD
...
Before, when withReplacement was set to true, we would not get a sample
bigger than the RDD's count().
Conflicts:
core/src/main/scala/spark/RDD.scala
core/src/test/scala/spark/RDDSuite.scala
2013-07-05 11:15:13 -07:00
Andrew xia
6ccfb73ca9
Add fair scheduler config template file
2013-07-04 19:19:44 +08:00
Patrick Wendell
8ca1cc1786
Adding truncation for log files
2013-07-02 16:10:50 -07:00
Patrick Wendell
8688689387
Various formatting changes
2013-07-01 13:40:12 -07:00