ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3684 commits 6 branches 181 tags 397 MiB
Commit graph

3684 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andrew xia 3b5a11e765 change function name "setName" to "setProperties" as "setName" is also member of Thread class 2013-08-01 19:37:15 +08:00
Dmitriy Lyubimov d29ee3689b Merge fixes merge commit hasn't picked 2013-08-01 00:21:26 -07:00
shivaram 58756b72f1 Merge pull request #761 from mateiz/kmeans-generator 
Add data generator for K-means
 2013-07-31 23:45:41 -07:00
Matei Zaharia 5ac548397d Fix string parsing and style in LR 2013-07-31 23:12:30 -07:00
Matei Zaharia 52dba89261 Turn on caching in KMeans.main 2013-07-31 23:08:12 -07:00
Dmitriy Lyubimov cb6be5bd7e Merge remote-tracking branch 'mesos/master' into SPARK-826 
Conflicts:
	core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala
	core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala
	core/src/test/scala/spark/KryoSerializerSuite.scala
 2013-07-31 22:09:22 -07:00
Dmitriy Lyubimov 28f1550f01 More elegant rewrite of the same. 2013-07-31 21:41:00 -07:00
Dmitriy Lyubimov 7c52ecc6a4 (1) added reduce test case. 
(2) added nested streaming in ParallelCollectionRDD
(3) added kryo with fold test which still doesn't work
 2013-07-31 19:27:30 -07:00
Matei Zaharia 3097d75d6f Merge remote-tracking branch 'dlyubimov/SPARK-827' 
Conflicts:
	docs/configuration.md
 2013-07-31 18:36:43 -07:00
Matei Zaharia ecab635ae1 Merge pull request #763 from c0s/assembly 
SPARK-842. Maven assembly is including examples libs and dependencies
 2013-07-31 18:16:55 -07:00
Konstantin Boudnik a6f43a97d1 SPARK-842. Maven assembly is including examples libs and dependencies 2013-07-31 17:26:55 -07:00
Karen Feng 7c9c5ef6c6 Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update 2013-07-31 16:39:26 -07:00
Karen Feng 02cde8efdf Replaces theme with Bootswatch Spacelab theme 2013-07-31 16:34:07 -07:00
Karen Feng 09cd67bf98 Changed bootstrap colors, fixed logpaging buttons 2013-07-31 16:18:53 -07:00
Karen Feng ef1f22bda9 Merge branch 'master' of https://github.com/mesos/spark 2013-07-31 16:13:49 -07:00
Matei Zaharia 39c75f3033 Merge pull request #757 from BlackNiuza/result_task_generation 
Bug fix: SPARK-837
 2013-07-31 15:52:36 -07:00
Matei Zaharia b2b86c2575 Merge pull request #753 from shivaram/glm-refactor 
Build changes for ML lib
 2013-07-31 15:51:39 -07:00
Matei Zaharia f607ffb9e1 Added data generator for K-means 
Also made it possible to specify the number of runs in KMeans.main().
 2013-07-31 14:31:07 -07:00
Matei Zaharia 14bf2fe039 Merge pull request #749 from benh/spark-executor-uri 
Added property 'spark.executor.uri' for launching on Mesos.
 2013-07-31 14:18:16 -07:00
Shivaram Venkataraman cef178873b Refactor SGD options into a new class. 
This refactoring pulls out code shared between SVM, Lasso, LR into
a common GradientDescentOpts class. Some style cleanup as well
 2013-07-31 14:15:17 -07:00
shivaram 4ba4c3fe1a Merge pull request #759 from mateiz/split-fix 
Use the Char version of split() instead of the String one in MLUtils
 2013-07-31 13:14:49 -07:00
Benjamin Hindman 529ac81195 Do not try and use 'scala' in 'run' from within a "release". 2013-07-31 12:50:12 -07:00
Benjamin Hindman 4692ea4892 Used 'uri.split('/').last' instead of 'new File(uri).getName()'. 2013-07-31 12:29:44 -07:00
Karen Feng c453967f9a Reduced size of heading 2013-07-31 11:57:50 -07:00
Matei Zaharia 9a444cffe7 Use the Char version of split() instead of the String one for efficiency 2013-07-31 11:28:39 -07:00
Matei Zaharia a386ced2c6 Merge pull request #754 from rxin/compression 
Compression codec change
 2013-07-31 11:22:50 -07:00
Karen Feng 49e6344142 Removed master URL from job UI, reduced heading size of basic spark pages 2013-07-31 11:17:59 -07:00
Reynold Xin 0be071abfc Merge pull request #756 from cdshines/patch-1 
Refactored Vector.apply(length, initializer) replacing excessive code with library method
 2013-07-31 11:11:59 -07:00
Reynold Xin c61843a69f Changed other LZF uses to use the compression codec interface. 2013-07-31 10:32:13 -07:00
Patrick Wendell 89da9d94b3 Add JSON path to master index page 2013-07-31 09:47:53 -07:00
BlackNiuza 9a815de4bf write and read generation in ResultTask 2013-08-01 00:36:47 +08:00
Roman Tkalenko 0c6553714a Refactored Vector.apply(length, initializer) replacing excessive code with library method 
(also removed unused variable ```ans``` as minor change)
 2013-07-31 19:05:46 +03:00
Matei Zaharia 12553e5c55 Simplified nonNegativeMod to match previous version 2013-07-31 08:50:28 -07:00
Matei Zaharia d4556f4207 Merge pull request #751 from cdshines/master 
Cleaned Partitioner & PythonPartitioner source by taking out non-related logic to Utils
 2013-07-31 08:48:14 -07:00
Andrew xia 5670c96f29 Merge branch 'master' into Pool_UI 
Conflicts:
	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/scheduler/DAGScheduler.scala
	core/src/main/scala/spark/scheduler/SparkListener.scala
	core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala
	core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala
	core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala
	core/src/main/scala/spark/ui/jobs/IndexPage.scala
	core/src/main/scala/spark/ui/jobs/JobProgressUI.scala
 2013-07-31 19:36:36 +08:00
cdshines fefb03cbd7 Eliminated code duplication, refactored to pattern-matching style Partitioner and PythonPartitioner 2013-07-31 13:19:42 +03:00
Dmitriy Lyubimov 96664431cb IDEA flipped JavaSerialized import at some point to a wrong class. 2013-07-30 23:10:09 -07:00
Dmitriy Lyubimov c219fc94fd Minor, style 2013-07-30 22:08:39 -07:00
Matei Zaharia 29b8cd3616 Merge pull request #755 from jerryshao/add-apache-header 
Add Apache license header to metrics system
 2013-07-30 21:30:33 -07:00
Dmitriy Lyubimov f4b4b8836e reverting back to one-by-one serialization for parallelize() 2013-07-30 19:00:58 -07:00
jerryshao bf9318091a Add Apache license header to metrics system 2013-07-31 09:42:16 +08:00
Shivaram Venkataraman 15fd0d619d Add mllib, bagel to repl dependencies 
Also don't build an assembly jar for them
 2013-07-30 18:31:11 -07:00
Reynold Xin 98024eadc3 Renamed compressionOutputStream and compressionInputStream to compressedOutputStream and compressedInputStream. 2013-07-30 18:28:46 -07:00
Dmitriy Lyubimov abada94ebf removing default constructor (not Externalizable any more) 2013-07-30 18:04:02 -07:00
Dmitriy Lyubimov 943c6590c9 realiging "extends" back manually 2013-07-30 18:01:35 -07:00
Dmitriy Lyubimov ca33b12e98 resetting wrap and continuation indent = 4 2013-07-30 17:51:44 -07:00
Reynold Xin dae12fef9e Updated the configuration option for Snappy block size to be consistent with the documentation. 2013-07-30 17:49:31 -07:00
Dmitriy Lyubimov 483d8dd694 Reverting SparkBuild w.r.t to branch fork ratherh than current trunk to avoid diffs appearing 2013-07-30 17:41:24 -07:00
Dmitriy Lyubimov 7ef2551af0 reverting build file..again 2013-07-30 17:39:43 -07:00
Dmitriy Lyubimov 984b56155a changing approaches for parallelize(): java serialization needs to avoid writing headers! 2013-07-30 17:36:59 -07:00