Dmitriy Lyubimov
ca33b12e98
resetting wrap and continuation indent = 4
2013-07-30 17:51:44 -07:00
Reynold Xin
dae12fef9e
Updated the configuration option for Snappy block size to be consistent with the documentation.
2013-07-30 17:49:31 -07:00
Dmitriy Lyubimov
483d8dd694
Reverting SparkBuild w.r.t to branch fork ratherh than current trunk to avoid diffs appearing
2013-07-30 17:41:24 -07:00
Dmitriy Lyubimov
7ef2551af0
reverting build file..again
2013-07-30 17:39:43 -07:00
Dmitriy Lyubimov
984b56155a
changing approaches for parallelize(): java serialization needs to avoid writing headers!
2013-07-30 17:36:59 -07:00
Reynold Xin
311aae76a2
Added Snappy dependency to Maven build files.
2013-07-30 17:25:42 -07:00
Reynold Xin
3b1ced83fb
Exclude older version of Snappy in streaming and examples.
2013-07-30 17:25:36 -07:00
Reynold Xin
56774b176e
Added unit test for compression codecs.
2013-07-30 17:12:33 -07:00
Reynold Xin
5227043f84
Documentation update for compression codec.
2013-07-30 17:12:16 -07:00
Reynold Xin
ad7e9d0d64
CompressionCodec cleanup. Moved it to spark.io package.
2013-07-30 17:11:54 -07:00
Benjamin Hindman
7bdafa918a
Format cleanup.
2013-07-30 17:01:00 -07:00
Dmitriy Lyubimov
ef9529a943
refactoring using writeByteBuffer() from Utils.
2013-07-30 16:24:23 -07:00
Dmitriy Lyubimov
43394b9a6d
fixing formatting
2013-07-30 16:13:41 -07:00
Dmitriy Lyubimov
13a9d66645
adding ===
2013-07-30 16:10:55 -07:00
Reynold Xin
368c58eac5
Merge branch 'lazy_file_open' of github.com:lyogavin/spark into compression
...
Conflicts:
project/SparkBuild.scala
2013-07-30 16:04:18 -07:00
Patrick Wendell
e87de037d6
Merge pull request #744 from karenfeng/bootstrap-update
...
Use Bootstrap progress bars in web UI
2013-07-30 15:00:08 -07:00
shivaram
ae57020598
Merge pull request #752 from rxin/master
...
Minor mllib cleanup
2013-07-30 14:56:41 -07:00
Shivaram Venkataraman
48851d4dd9
Add bagel, mllib to SBT assembly.
...
Also add jblas dependency to mllib pom.xml
2013-07-30 14:03:15 -07:00
Reynold Xin
366f7735eb
Minor style cleanup of mllib.
2013-07-30 13:59:32 -07:00
Reynold Xin
47011e6854
Use a tigher bound in logistic regression unit test's prediction validation.
2013-07-30 13:58:23 -07:00
Reynold Xin
e35966ae9a
Renamed Classification.scala to ClassificationModel.scala and Regression.scala to RegressionModel.scala
2013-07-30 13:28:31 -07:00
Karen Feng
26144c400f
Fixed wrap style
2013-07-30 12:40:41 -07:00
Karen Feng
218d7c4ed8
Fixed style, lowered height of progress bars
2013-07-30 12:39:17 -07:00
Karen Feng
f1cab31b73
Removed intermediate set for activeTasks, removed progress bar margin
2013-07-30 11:06:47 -07:00
Dmitriy Lyubimov
1bca91633e
+ bug fixes;
...
test added
Conflicts:
core/src/test/scala/spark/KryoSerializerSuite.scala
2013-07-30 11:04:11 -07:00
shivaram
8aee118216
Merge pull request #748 from atalwalkar/master
...
made SimpleUpdater consistent with other updaters
2013-07-30 10:27:54 -07:00
Benjamin Hindman
f6f46455eb
Added property 'spark.executor.uri' for launching on Mesos without
...
requiring Spark to be installed. Using 'make_distribution.sh' a user
can put a Spark distribution at a URI supported by Mesos (e.g.,
'hdfs://...') and then set that when launching their job. Also added
SPARK_EXECUTOR_URI for the REPL.
2013-07-29 23:32:52 -07:00
Ameet Talwalkar
e4387ddf5d
made SimpleUpdater consistent with other updaters
2013-07-29 22:21:50 -07:00
Josh Rosen
49be084ed3
Use File.pathSeparator instead of hardcoding ':'.
2013-07-29 22:08:57 -07:00
Josh Rosen
b95732632b
Do not inherit master's PYTHONPATH on workers.
...
This fixes SPARK-832, an issue where PySpark
would not work when the master and workers used
different SPARK_HOME paths.
This change may potentially break code that relied
on the master's PYTHONPATH being used on workers.
To have custom PYTHONPATH additions used on the
workers, users should set a custom PYTHONPATH in
spark-env.sh rather than setting it in the shell.
2013-07-29 22:08:57 -07:00
Andrew xia
5406013997
refactor codes less than 100 character per line
2013-07-30 11:41:38 +08:00
Andrew xia
614ee16cc4
refactor job ui with pool information
2013-07-30 10:57:26 +08:00
Matei Zaharia
468a36c005
Merge pull request #746 from rxin/cleanup
...
Internal cleanup
2013-07-29 19:44:33 -07:00
atalwalkar
1e1ffb192a
Merge pull request #745 from shivaram/loss-update-fix
...
Remove duplicate loss history in Gradient Descent
2013-07-29 19:26:19 -07:00
Matei Zaharia
01f94931d5
Update the Python logistic regression example to read from a file and
...
batch input records for more efficient NumPy computations
2013-07-29 19:23:41 -07:00
Dmitriy Lyubimov
8e5cd041bb
initial externalization of ParallelCollectionRDD's split
2013-07-29 19:02:53 -07:00
Shivaram Venkataraman
3ca9faa341
Clarify how regVal is computed in Updater docs
2013-07-29 18:37:28 -07:00
Reynold Xin
81720e13fc
Moved all StandaloneClusterMessage's into StandaloneClusterMessages object.
2013-07-29 17:53:01 -07:00
Reynold Xin
23b5da14ed
Moved block manager messages into BlockManagerMessages object.
2013-07-29 17:42:05 -07:00
Reynold Xin
105f4d22e9
Removed Cache and SoftReferenceCache since they are no longer used.
2013-07-29 17:30:38 -07:00
Matei Zaharia
207548b67b
Open up Job UI ports (33000-33010) on EC2 clusters
2013-07-29 17:19:33 -07:00
Reynold Xin
17e62113d4
Moved DeployMessage's into its own DeployMessages object.
...
Also renamed MasterState to MasterStateResponse and WorkerState to WorkerStateResponse for clarity.
2013-07-29 17:14:44 -07:00
Karen Feng
87b821dc39
Fixed continuity of executorToTasksActive, changed color of progress bars
2013-07-29 16:50:51 -07:00
Karen Feng
c7b2788948
Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update
...
Conflicts:
core/src/main/scala/spark/ui/jobs/IndexPage.scala
2013-07-29 16:36:07 -07:00
Patrick Wendell
c99b674405
Merge pull request #735 from karenfeng/ui-807
...
Totals for shuffle data and CPU time
2013-07-29 16:32:55 -07:00
Shivaram Venkataraman
07da72b451
Remove duplicate loss history and clarify why.
...
Also some minor style fixes.
2013-07-29 16:25:17 -07:00
Karen Feng
2d6da9195a
Alphabetized imports
2013-07-29 15:50:52 -07:00
Karen Feng
478a2886d9
Added started tasks to progress bar
2013-07-29 14:51:07 -07:00
Karen Feng
e04a37a332
Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update
...
cially if it merges an updated upstream into a topic branch.
2013-07-29 14:32:48 -07:00
Reynold Xin
fe7298b587
Merge pull request #741 from pwendell/usability
...
Fix two small usability issues
2013-07-29 14:01:00 -07:00