ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3295 commits 6 branches 181 tags 397 MiB
Commit graph

3295 commits

This branch
This branch
All branches
Author SHA1 Message Date
BlackNiuza c1d44be805 Bug fix: SPARK-796 2013-07-09 15:18:28 +08:00
Matei Zaharia 7dcda9ae74 Merge pull request #688 from markhamstra/scalaDependencies 
Fixed SPARK-795 with explicit dependencies
 2013-07-08 23:24:23 -07:00
Matei Zaharia 638927b78e Merge pull request #683 from shivaram/sbt-test-fix 
Remove some stack traces from sbt test output
 2013-07-08 22:58:50 -07:00
Mark Hamstra 0b39d66f3f pom cleanup 2013-07-08 16:07:09 -07:00
Mark Hamstra afdaf430bd Explicit dependencies for scala-library and scalap to prevent 2.9.2 vs. 2.9.3 problems 2013-07-08 15:40:50 -07:00
shivaram 3c1317835e Merge pull request #687 from atalwalkar/master 
Added "Labeled" to util functions for labeled data
 2013-07-08 14:50:34 -07:00
Ameet Talwalkar bf4c9a5e0f renamed with labeled prefix 2013-07-08 14:37:42 -07:00
Charles Reiss 8c1d1c98e0 Explicitly set class loader for MesosSchedulerDriver callbacks. 2013-07-08 12:25:46 -07:00
shivaram 744da8eefd Merge pull request #679 from ryanlecompte/master 
Make binSearch method tail-recursive for RidgeRegression
 2013-07-07 17:42:25 -07:00
ryanlecompte be123aa6ef update to use ListBuffer, faster than Vector for append operations 2013-07-07 15:35:06 -07:00
Shivaram Venkataraman 4af0d63cb1 Remove akka LogLevel fix as we no longer use spray 2013-07-07 10:42:43 -07:00
Shivaram Venkataraman d362d0f411 Ignore stderr when calling cat on a non-existing file 2013-07-07 04:09:46 -07:00
Shivaram Venkataraman 3350ad0d7f Catch RejectedExecution exception in Checkpoint handler. 2013-07-07 04:09:37 -07:00
Shivaram Venkataraman 7d6d9e6ab2 Set DriverSuite log level to WARN 2013-07-07 04:09:15 -07:00
Shivaram Venkataraman a948f06725 Suppress log messages in sbt test with two changes: 
1. Set akka log level to ERROR before shutting down the actorSystem.
This avoids akka log messages (like Spray) from falling back to INFO
on the Stdout logger
2. Initialize netty to use SLF4J in LocalSparkContext. This ensures that
stack trace thrown during shutdown is handled by SLF4J instead of stdout
 2013-07-07 04:09:08 -07:00
Matei Zaharia 3cc6818f13 Merge pull request #668 from shimingfei/guava-14.0.1 
update guava version from 11.0.1 to 14.0.1
 2013-07-06 19:51:20 -07:00
ryanlecompte f78f8d0b41 fix formatting and use Vector instead of List to maintain order 2013-07-06 16:46:53 -07:00
Matei Zaharia ebe1efc862 Merge remote-tracking branch 'pwendell/ui-updates' 2013-07-06 16:46:15 -07:00
Matei Zaharia fd6665122b Fix some other references to Cloudera Avro and updated Avro version 2013-07-06 16:45:15 -07:00
Patrick Wendell 32b9d21a97 Fix occasional failure in UI listener. 
If a task fails before the metrics are initialized, it remains possible
that the metrics field will be `None`. This patch accounts for that possbility
by keeping metrics as an `Option` at all times.
 2013-07-06 16:40:02 -07:00
Matei Zaharia 22161887ee Merge pull request #676 from c0s/asf-avro 
Use standard ASF published avro module instead of a proprietory built one
 2013-07-06 16:18:15 -07:00
Matei Zaharia 1ffadb2d9e Merge remote-tracking branch 'pwendell/ui-updates' 
Conflicts:
	core/src/main/scala/spark/scheduler/DAGScheduler.scala
	core/src/main/scala/spark/util/AkkaUtils.scala
	pom.xml
 2013-07-06 15:51:41 -07:00
Matei Zaharia 94871e4703 Merge pull request #655 from tgravescs/master 
Add support for running Spark on Yarn on a secure Hadoop Cluster
 2013-07-06 15:26:19 -07:00
Matei Zaharia 3f918b33f8 Merge pull request #672 from holdenk/master 
s/ActorSystemImpl/ExtendedActorSystem/ as ActorSystemImpl results in a warning
 2013-07-06 12:45:18 -07:00
Matei Zaharia 2a36e5449b Merge pull request #673 from xiajunluan/master 
Add config template file for fair scheduler feature
 2013-07-06 12:43:21 -07:00
Matei Zaharia 7ba7fa110b Merge pull request #674 from liancheng/master 
Bug fix: SPARK-789
 2013-07-06 11:45:08 -07:00
Matei Zaharia f4416a1d7e Merge pull request #681 from BlackNiuza/memory_leak 
Remove active job from idToActiveJob when job finished or aborted
 2013-07-06 11:41:58 -07:00
BlackNiuza 44a2440039 Remove active job from idToActiveJob when job finished or aborted 2013-07-07 01:33:09 +08:00
Patrick Wendell 37abe84212 Tracking some task metrics even during failures. 2013-07-06 09:19:59 -07:00
Matei Zaharia e063e29af8 Merge pull request #680 from tdas/master 
Fixed major performance bug in Network Receiver
 2013-07-05 21:54:52 -07:00
Tathagata Das 280418ac45 Reduced the number of Iterator to ArrayBuffer copies in NetworkReceiver. 2013-07-05 21:38:21 -07:00
ryanlecompte 757e56dfc7 make binSearch a tail-recursive method 2013-07-05 19:54:28 -07:00
shivaram bf1311e6d2 Merge pull request #678 from mateiz/ml-examples 
Start of ML package
 2013-07-05 17:32:44 -07:00
Matei Zaharia 8bbe907556 Replaced string constants in test 2013-07-05 17:25:23 -07:00
Patrick Wendell 84b7fc54e6 Enforcing correct sort order for formatted strings 2013-07-05 17:21:08 -07:00
Matei Zaharia 653043beb6 Renamed files to match package 2013-07-05 17:18:55 -07:00
Matei Zaharia de67deeaab Addressed style comments from Ryan LeCompte 2013-07-05 17:16:49 -07:00
Matei Zaharia 43b24635ee Renamed ML package to MLlib and added it to classpath 2013-07-05 11:38:53 -07:00
Matei Zaharia 399bd65ef5 Fixed compile error due to merge 2013-07-05 11:27:06 -07:00
Shivaram Venkataraman 0e33c88cbd Rename package gradient to optimization 2013-07-05 11:15:19 -07:00
Shivaram Venkataraman 09f187a400 Add top-level methods for regression methods. 
Also add multiple versions of them to make it easier to call them from java.
 2013-07-05 11:15:19 -07:00
Matei Zaharia 9441d3ef09 Use random seeds for K-means and ALS, and increase tolerance in tests 
Random seeds make more sense by default for a machine learning library
because other libraries behave the same way (people expect to be able to
run the algorithm multiple times and get a better answer), but we can
add configuration later if needed. Tests that depend on specific seed
choices seem brittle.
 2013-07-05 11:15:19 -07:00
Matei Zaharia e7d49388e3 Added unit test for K-means, and fixed some bugs 2013-07-05 11:15:19 -07:00
Matei Zaharia 652ea0f1d8 Allow RDD.takeSample to give samples bigger than the RDD 
Before, when withReplacement was set to true, we would not get a sample
bigger than the RDD's count().

Conflicts:
	core/src/main/scala/spark/RDD.scala
	core/src/test/scala/spark/RDDSuite.scala
 2013-07-05 11:15:13 -07:00
Matei Zaharia cffe3340c5 Fix logistic regression test failure and test suite cleanup 2013-07-05 11:13:46 -07:00
Shivaram Venkataraman 496c7548bb Change test to use fewer iterations 2013-07-05 11:13:46 -07:00
Matei Zaharia 52f491125e Implementation of k-means and k-means|| 2013-07-05 11:13:46 -07:00
Matei Zaharia 39684eafe3 Formatting 2013-07-05 11:13:46 -07:00
Matei Zaharia 6586c5e28b Added a SparkContext accessor to RDD 2013-07-05 11:13:46 -07:00
Matei Zaharia 43dae967d7 Renamed "als" package to "recommendation" 2013-07-05 11:13:46 -07:00