ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3465 commits 6 branches 181 tags 397 MiB
Commit graph

3465 commits

This branch
This branch
All branches
Author SHA1 Message Date
Dmitriy Lyubimov cb6be5bd7e Merge remote-tracking branch 'mesos/master' into SPARK-826 
Conflicts:
	core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala
	core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala
	core/src/test/scala/spark/KryoSerializerSuite.scala
 2013-07-31 22:09:22 -07:00
Dmitriy Lyubimov 28f1550f01 More elegant rewrite of the same. 2013-07-31 21:41:00 -07:00
Dmitriy Lyubimov 7c52ecc6a4 (1) added reduce test case. 
(2) added nested streaming in ParallelCollectionRDD
(3) added kryo with fold test which still doesn't work
 2013-07-31 19:27:30 -07:00
Matei Zaharia 3097d75d6f Merge remote-tracking branch 'dlyubimov/SPARK-827' 
Conflicts:
	docs/configuration.md
 2013-07-31 18:36:43 -07:00
Matei Zaharia ecab635ae1 Merge pull request #763 from c0s/assembly 
SPARK-842. Maven assembly is including examples libs and dependencies
 2013-07-31 18:16:55 -07:00
Konstantin Boudnik a6f43a97d1 SPARK-842. Maven assembly is including examples libs and dependencies 2013-07-31 17:26:55 -07:00
Matei Zaharia 39c75f3033 Merge pull request #757 from BlackNiuza/result_task_generation 
Bug fix: SPARK-837
 2013-07-31 15:52:36 -07:00
Matei Zaharia b2b86c2575 Merge pull request #753 from shivaram/glm-refactor 
Build changes for ML lib
 2013-07-31 15:51:39 -07:00
Matei Zaharia 14bf2fe039 Merge pull request #749 from benh/spark-executor-uri 
Added property 'spark.executor.uri' for launching on Mesos.
 2013-07-31 14:18:16 -07:00
shivaram 4ba4c3fe1a Merge pull request #759 from mateiz/split-fix 
Use the Char version of split() instead of the String one in MLUtils
 2013-07-31 13:14:49 -07:00
Benjamin Hindman 529ac81195 Do not try and use 'scala' in 'run' from within a "release". 2013-07-31 12:50:12 -07:00
Benjamin Hindman 4692ea4892 Used 'uri.split('/').last' instead of 'new File(uri).getName()'. 2013-07-31 12:29:44 -07:00
Matei Zaharia 9a444cffe7 Use the Char version of split() instead of the String one for efficiency 2013-07-31 11:28:39 -07:00
Matei Zaharia a386ced2c6 Merge pull request #754 from rxin/compression 
Compression codec change
 2013-07-31 11:22:50 -07:00
Reynold Xin 0be071abfc Merge pull request #756 from cdshines/patch-1 
Refactored Vector.apply(length, initializer) replacing excessive code with library method
 2013-07-31 11:11:59 -07:00
Reynold Xin c61843a69f Changed other LZF uses to use the compression codec interface. 2013-07-31 10:32:13 -07:00
BlackNiuza 9a815de4bf write and read generation in ResultTask 2013-08-01 00:36:47 +08:00
Roman Tkalenko 0c6553714a Refactored Vector.apply(length, initializer) replacing excessive code with library method 
(also removed unused variable ```ans``` as minor change)
 2013-07-31 19:05:46 +03:00
Matei Zaharia 12553e5c55 Simplified nonNegativeMod to match previous version 2013-07-31 08:50:28 -07:00
Matei Zaharia d4556f4207 Merge pull request #751 from cdshines/master 
Cleaned Partitioner & PythonPartitioner source by taking out non-related logic to Utils
 2013-07-31 08:48:14 -07:00
cdshines fefb03cbd7 Eliminated code duplication, refactored to pattern-matching style Partitioner and PythonPartitioner 2013-07-31 13:19:42 +03:00
Dmitriy Lyubimov 96664431cb IDEA flipped JavaSerialized import at some point to a wrong class. 2013-07-30 23:10:09 -07:00
Dmitriy Lyubimov c219fc94fd Minor, style 2013-07-30 22:08:39 -07:00
Matei Zaharia 29b8cd3616 Merge pull request #755 from jerryshao/add-apache-header 
Add Apache license header to metrics system
 2013-07-30 21:30:33 -07:00
Dmitriy Lyubimov f4b4b8836e reverting back to one-by-one serialization for parallelize() 2013-07-30 19:00:58 -07:00
jerryshao bf9318091a Add Apache license header to metrics system 2013-07-31 09:42:16 +08:00
Shivaram Venkataraman 15fd0d619d Add mllib, bagel to repl dependencies 
Also don't build an assembly jar for them
 2013-07-30 18:31:11 -07:00
Reynold Xin 98024eadc3 Renamed compressionOutputStream and compressionInputStream to compressedOutputStream and compressedInputStream. 2013-07-30 18:28:46 -07:00
Dmitriy Lyubimov abada94ebf removing default constructor (not Externalizable any more) 2013-07-30 18:04:02 -07:00
Dmitriy Lyubimov 943c6590c9 realiging "extends" back manually 2013-07-30 18:01:35 -07:00
Dmitriy Lyubimov ca33b12e98 resetting wrap and continuation indent = 4 2013-07-30 17:51:44 -07:00
Reynold Xin dae12fef9e Updated the configuration option for Snappy block size to be consistent with the documentation. 2013-07-30 17:49:31 -07:00
Dmitriy Lyubimov 483d8dd694 Reverting SparkBuild w.r.t to branch fork ratherh than current trunk to avoid diffs appearing 2013-07-30 17:41:24 -07:00
Dmitriy Lyubimov 7ef2551af0 reverting build file..again 2013-07-30 17:39:43 -07:00
Dmitriy Lyubimov 984b56155a changing approaches for parallelize(): java serialization needs to avoid writing headers! 2013-07-30 17:36:59 -07:00
Reynold Xin 311aae76a2 Added Snappy dependency to Maven build files. 2013-07-30 17:25:42 -07:00
Reynold Xin 3b1ced83fb Exclude older version of Snappy in streaming and examples. 2013-07-30 17:25:36 -07:00
Reynold Xin 56774b176e Added unit test for compression codecs. 2013-07-30 17:12:33 -07:00
Reynold Xin 5227043f84 Documentation update for compression codec. 2013-07-30 17:12:16 -07:00
Reynold Xin ad7e9d0d64 CompressionCodec cleanup. Moved it to spark.io package. 2013-07-30 17:11:54 -07:00
Benjamin Hindman 7bdafa918a Format cleanup. 2013-07-30 17:01:00 -07:00
Dmitriy Lyubimov ef9529a943 refactoring using writeByteBuffer() from Utils. 2013-07-30 16:24:23 -07:00
Dmitriy Lyubimov 43394b9a6d fixing formatting 2013-07-30 16:13:41 -07:00
Dmitriy Lyubimov 13a9d66645 adding === 2013-07-30 16:10:55 -07:00
Reynold Xin 368c58eac5 Merge branch 'lazy_file_open' of github.com:lyogavin/spark into compression 
Conflicts:
	project/SparkBuild.scala
 2013-07-30 16:04:18 -07:00
Patrick Wendell e87de037d6 Merge pull request #744 from karenfeng/bootstrap-update 
Use Bootstrap progress bars in web UI
 2013-07-30 15:00:08 -07:00
shivaram ae57020598 Merge pull request #752 from rxin/master 
Minor mllib cleanup
 2013-07-30 14:56:41 -07:00
Shivaram Venkataraman 48851d4dd9 Add bagel, mllib to SBT assembly. 
Also add jblas dependency to mllib pom.xml
 2013-07-30 14:03:15 -07:00
Reynold Xin 366f7735eb Minor style cleanup of mllib. 2013-07-30 13:59:32 -07:00
Reynold Xin 47011e6854 Use a tigher bound in logistic regression unit test's prediction validation. 2013-07-30 13:58:23 -07:00