Joseph E. Gonzalez
61281756f2
IndexedRDD passes all PairRDD Function tests
2013-08-15 14:20:59 -07:00
Joseph E. Gonzalez
54b54903c3
Adding testing code for indexedrdd
2013-08-14 16:35:20 -07:00
Joseph E. Gonzalez
b71d4febbc
Finished early prototype of IndexedRDD
2013-08-14 15:25:56 -07:00
Joseph E. Gonzalez
f2b8dd3929
second indexedrdd design
2013-08-13 14:21:49 -07:00
Evan Sparks
4346f0a1e9
Merge pull request #809 from shivaram/sgd-cleanup
...
Clean up scaladoc in ML Lib.
2013-08-12 12:12:12 -07:00
Matei Zaharia
ea1b4baabd
Merge pull request #806 from apivovarov/yarn-205
...
Changed yarn.version to 2.0.5 in pom.xml
2013-08-12 08:09:58 -07:00
Shivaram Venkataraman
8b5e3e2eb5
Add ML Lib scaladoc to API dropdown
2013-08-11 23:52:43 -07:00
Reynold Xin
2a39d2ca25
Merge pull request #810 from pwendell/dead_doc_code
...
Remove now dead code inside of docs
2013-08-11 20:35:09 -07:00
Patrick Wendell
9244524146
Removing dead docs
2013-08-11 20:33:58 -07:00
Shivaram Venkataraman
4935a2558b
Clean up scaladoc in ML Lib.
...
Also build and copy ML Lib scaladoc in Spark docs build.
Some more minor cleanup with respect to naming, test locations etc.
2013-08-11 19:02:43 -07:00
Reynold Xin
e5b9ed2833
Merge pull request #808 from pwendell/ui_compressed_bytes
...
Report compressed bytes read when calculating TaskMetrics
2013-08-11 17:22:47 -07:00
Patrick Wendell
3d8f281604
Report compressed bytes read when calculating TaskMetrics
2013-08-11 16:25:57 -07:00
Matei Zaharia
379648630b
Merge pull request #805 from woggle/hadoop-rdd-jobconf
...
Use new Configuration() instead of slower new JobConf() in SerializableWritable
2013-08-11 14:51:47 -07:00
Evan Sparks
ff9ebfabb4
Merge pull request #762 from shivaram/sgd-cleanup
...
Refactor SGD options into a new class.
2013-08-11 10:52:55 -07:00
shivaram
95c62ca306
Merge pull request #804 from apivovarov/master
...
Fixed path to JavaALS.java and JavaKMeans.java, fixed hadoop2-yarn profi...
2013-08-11 10:30:52 -07:00
Alexander Pivovarov
2d97cc46af
Fixed path to JavaALS.java and JavaKMeans.java, fixed hadoop2-yarn profile
2013-08-10 23:04:50 -07:00
Alexander Pivovarov
ca28f2e639
Changed yarn.version to 2.0.5 in pom.xml
2013-08-10 22:50:04 -07:00
Charles Reiss
6402b539d0
Use new Configuration() instead of new JobConf() for ObjectWritable.
...
JobConf's constructor loads default config files in some verisons of
Hadoop, which is quite slow, and we only need the Configuration object
to pass the correct ClassLoader.
2013-08-10 21:31:05 -07:00
Shivaram Venkataraman
a65a6ed514
Fix GLM code review comments and move java tests
2013-08-10 18:54:10 -07:00
Matei Zaharia
b13dae3ac6
Add a sample data file for PageRank
2013-08-10 18:13:49 -07:00
Matei Zaharia
4c4f769187
Optimize Scala PageRank to use reduceByKey
2013-08-10 18:09:54 -07:00
Matei Zaharia
06e4f2a8f2
Merge pull request #789 from MLnick/master
...
Adding Scala version of PageRank example
2013-08-10 18:06:23 -07:00
Matei Zaharia
71c63de22f
Merge pull request #795 from mridulm/master
...
Fix bug reported in PR 791 : a race condition in ConnectionManager and Connection
2013-08-10 10:21:20 -07:00
Matei Zaharia
d3277a0daf
Merge remote-tracking branch 'origin/pr/792'
...
Conflicts:
core/src/main/scala/spark/ui/jobs/IndexPage.scala
core/src/main/scala/spark/ui/jobs/StagePage.scala
2013-08-10 10:18:50 -07:00
Patrick Wendell
d17eeb997d
Merge pull request #785 from anfeng/master
...
expose HDFS file system stats via Executor metrics
2013-08-10 09:02:27 -07:00
Kay Ousterhout
14d14f451a
Shortened names, as per Matei's suggestion
2013-08-10 07:50:27 -07:00
Matei Zaharia
dce5e47435
Merge pull request #800 from dlyubimov/HBASE_VERSION
...
Pull HBASE_VERSION in the head of sbt build
2013-08-09 21:53:45 -07:00
Matei Zaharia
cd247ba5bb
Merge pull request #786 from shivaram/mllib-java
...
Java fixes, tests and examples for ALS, KMeans
2013-08-09 20:41:13 -07:00
Kay Ousterhout
7810a76512
Only print event queue full error message once
2013-08-09 18:20:48 -07:00
Kay Ousterhout
44ca8629d8
Style fix: removing unnecessary return type
2013-08-09 17:22:50 -07:00
Kay Ousterhout
29b79714f9
Style fixes based on code review
2013-08-09 16:46:34 -07:00
Dmitriy Lyubimov
27f674f82b
fewer words
2013-08-09 13:54:41 -07:00
Kay Ousterhout
81e1d4a7d1
Refactored SparkListener to process all events asynchronously.
...
This commit fixes issues where SparkListeners that take a while to
process events slow the DAGScheduler.
This commit also fixes a bug in the UI where if a user goes to a
web page of a stage that does not exist, they can create a memory
leak (granted, this is not an issue at small scale -- probably only
an issue if someone actively tried to DOS the UI).
2013-08-09 13:27:41 -07:00
Matei Zaharia
b09d4b79e8
Merge pull request #799 from woggle/sync-fix
...
Remove extra synchronization in ResultTask
2013-08-09 13:17:08 -07:00
Patrick Wendell
cc6b92e80e
Merge pull request #775 from pwendell/print-launch-command
...
Log the launch command for Spark daemons
2013-08-09 13:00:33 -07:00
Dmitriy Lyubimov
ae95b57469
Pull HBASE_VERSION in the head of sbt build
2013-08-09 12:45:18 -07:00
Patrick Wendell
3970b580c2
Using quotes when printing out command
2013-08-09 11:53:32 -07:00
Charles Reiss
9dfc280f74
Remove extra synchronization in ResultTask
2013-08-09 11:09:02 -07:00
Matei Zaharia
f94fc75c3f
Merge pull request #788 from shane-huang/sparkjavaopts
...
For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as ...
2013-08-09 10:04:03 -07:00
shane-huang
34c1f8d2da
add comment in spark-env.sh.template for SPARK_JAVA_OPTS
...
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
2013-08-09 15:03:48 +08:00
Matei Zaharia
06303a62e5
Optimize JavaPageRank to use reduceByKey instead of groupByKey
2013-08-08 18:50:00 -07:00
Reynold Xin
01f20a941e
Fixed a typo in mllib inline documentation.
2013-08-08 16:42:54 -07:00
Shivaram Venkataraman
2812e72200
Add setters for optimizer, gradient in SGD.
...
Also remove java-specific constructor for LabeledPoint.
2013-08-08 16:24:31 -07:00
Shivaram Venkataraman
e1a209f791
Remove Java-specific constructor for Rating.
...
The scala constructor works for native type java types. Modify examples
to match this.
2013-08-08 14:36:02 -07:00
Matei Zaharia
63b6e02bed
Merge pull request #797 from mateiz/chill-0.3.1
...
Update to Chill 0.3.1
2013-08-08 14:02:02 -07:00
Matei Zaharia
d1e1c1b24d
Add test for Kryo with WrappedArray (which was failing in Chill 0.3.0)
2013-08-08 13:34:11 -07:00
Matei Zaharia
5a4003c1ac
Update to Chill 0.3.1
2013-08-08 13:30:27 -07:00
Patrick Wendell
9955e5a4b7
Merge pull request #796 from pwendell/bootstrap-design
...
Bootstrap re-design
2013-08-08 11:03:38 -07:00
Mridul Muralidharan
c230ca3b4e
Change line size
2013-08-08 22:28:40 +05:30
Mridul Muralidharan
dc47084f4e
Attempt to fix bug reported in PR 791 : a race condition in ConnectionManager and Connection
2013-08-08 22:19:27 +05:30