Reynold Xin
758ceff778
Updated BytecodeUtils to ASM4.
2013-06-29 15:39:48 -07:00
Reynold Xin
564d902d79
Merge branch 'master' of github.com:mesos/spark into graph
...
Conflicts:
run
run2.cmd
2013-06-29 15:30:21 -07:00
Reynold Xin
ae0eca5ec8
Merge branch 'graph' of github.com:rxin/spark into graph
2013-06-29 15:22:33 -07:00
Reynold Xin
ae12d163dc
Added the BytecodeUtils class for analyzing bytecode.
2013-06-29 15:22:15 -07:00
Joseph E. Gonzalez
a80b28a579
Renamed several functions and classes and improved documentation
2013-06-28 18:18:51 -07:00
Matei Zaharia
50ca17635a
Merge pull request #664 from pwendell/test-fix
...
Removing incorrect test statement
2013-06-27 22:24:52 -07:00
Matei Zaharia
4974b658ed
Look at JAVA_HOME before PATH to determine Java executable
2013-06-27 22:16:40 -07:00
Patrick Wendell
c767e74370
Removing incorrect test statement
2013-06-27 21:48:58 -07:00
Matei Zaharia
aea727f68d
Simplify Python docs a little to do substring search
2013-06-26 21:15:09 -07:00
Matei Zaharia
03906f7f0a
Fixes to compute-classpath on Windows
2013-06-26 17:40:22 -07:00
Matei Zaharia
e49bc8ca8c
Merge pull request #663 from stephenh/option_and_getenv
...
Be cute with Option and getenv.
2013-06-26 11:13:33 -07:00
Stephen Haberman
d7011632d1
Wrap lines.
2013-06-26 12:35:57 -05:00
Stephen Haberman
d11025dc6a
Be cute with Option and getenv.
2013-06-26 09:53:35 -05:00
Matei Zaharia
32370da4e4
Don't use forward slash in exclusion for JAR signature files
2013-06-25 22:08:19 -04:00
Matei Zaharia
9f0d913295
Refactored tests to share SparkContexts in some of them
...
Creating these seems to take a while and clutters the output with Akka
stuff, so it would be nice to share them.
2013-06-25 19:18:30 -04:00
Matei Zaharia
2bd04c3513
Formatting
2013-06-25 18:37:14 -04:00
Matei Zaharia
f2263350ed
Added a local-cluster mode test to ReplSuite
2013-06-25 18:35:35 -04:00
Matei Zaharia
6c8d1b2ca6
Fix computation of classpath when we launch java directly
...
The previous version assumed that a CLASSPATH environment variable was
set by the "run" script when launching the process that starts the
ExecutorRunner, but unfortunately this is not true in tests. Instead, we
factor the classpath calculation into an extenral script and call that.
NOTE: This includes a Windows version but hasn't yet been tested there.
2013-06-25 18:21:00 -04:00
Matei Zaharia
15b00914c5
Some fixes to the launch-java-directly change:
...
- Split SPARK_JAVA_OPTS into multiple command-line arguments if it
contains spaces; this splitting follows quoting rules in bash
- Add the Scala JARs to the classpath if they're not in the CLASSPATH
variable because the ExecutorRunner is launched with "scala" (this can
happen when using local-cluster URLs in spark-shell)
2013-06-25 17:17:27 -04:00
Matei Zaharia
7680ce0bd6
Fixed deprecated use of expect in SizeEstimatorSuite
2013-06-25 16:11:44 -04:00
Matei Zaharia
7e0191c6ea
Merge remote-tracking branch 'cgrothaus/SPARK-698'
...
Conflicts:
run
2013-06-25 15:47:40 -04:00
Matei Zaharia
f5e32ed13a
Merge pull request #661 from mesos/streaming
...
Kafka fixes and DStream.count fix for master
2013-06-25 09:16:57 -07:00
Tathagata Das
c89af0a7f9
Merge branch 'master' into streaming
...
Conflicts:
.gitignore
2013-06-24 23:57:47 -07:00
Tathagata Das
48c7e373c6
Minor formatting fixes
2013-06-24 23:11:04 -07:00
Tathagata Das
1249e9153b
Merge pull request #572 from Reinvigorate/sm-block-interval
...
Adding spark.streaming.blockInterval property
2013-06-24 21:46:33 -07:00
Tathagata Das
cfcda95f86
Merge pull request #571 from Reinvigorate/sm-kafka-serializers
...
Surfacing decoders on KafkaInputDStream
2013-06-24 21:44:50 -07:00
Tathagata Das
575aff6b71
Merge pull request #567 from Reinvigorate/sm-count-fix
...
Fixing count() in Spark Streaming
2013-06-24 21:35:50 -07:00
Matei Zaharia
78ffe164b3
Clone the zero value for each key in foldByKey
...
The old version reused the object within each task, leading to
overwriting of the object when a mutable type is used, which is expected
to be common in fold.
Conflicts:
core/src/test/scala/spark/ShuffleSuite.scala
2013-06-23 10:26:53 -07:00
Matei Zaharia
0e0f9d3069
Fix search path for REPL class loader to really find added JARs
2013-06-22 17:44:04 -07:00
Matei Zaharia
b5df1cd668
ADD_JARS environment variable for spark-shell
2013-06-22 17:14:44 -07:00
Matei Zaharia
3e61beff7b
Merge pull request #648 from shivaram/netty-dbg
...
Shuffle fixes and cleanup
2013-06-22 16:22:47 -07:00
Matei Zaharia
d92d3f7938
Fix resolution of example code with Maven builds
2013-06-22 10:24:19 -07:00
Matei Zaharia
1ef5d0d2c9
Merge pull request #644 from shimingfei/joblogger
...
add Joblogger to Spark (on new Spark code)
2013-06-22 09:35:57 -07:00
Matei Zaharia
7e4b266678
Merge pull request #563 from jey/python-optimization
...
Optimize PySpark worker invocation
2013-06-22 07:53:18 -07:00
Matei Zaharia
b350f34703
Increase memory for tests to prevent a crash on JDK 7
2013-06-22 07:48:20 -07:00
Jey Kottalam
c75bed0eeb
Fix reporting of PySpark exceptions
2013-06-21 12:14:16 -04:00
Jey Kottalam
1ba3c17303
use parens when calling method with side-effects
2013-06-21 12:14:16 -04:00
Jey Kottalam
7c5ff733ee
PySpark daemon: fix deadlock, improve error handling
2013-06-21 12:14:16 -04:00
Jey Kottalam
edb18ca928
Rename PythonWorker to PythonWorkerFactory
2013-06-21 12:14:16 -04:00
Jey Kottalam
62c4781400
Add tests and fixes for Python daemon shutdown
2013-06-21 12:14:16 -04:00
Jey Kottalam
c79a6078c3
Prefork Python worker processes
2013-06-21 12:14:16 -04:00
Jey Kottalam
40afe0d2a5
Add Python timing instrumentation
2013-06-21 12:14:16 -04:00
Mingfei
2fc794a6c7
small modify in DAGScheduler
2013-06-21 18:21:35 +08:00
Mingfei
4b9862ac9c
small format modification
2013-06-21 17:55:32 +08:00
Mingfei
aa7aa587be
some format modification
2013-06-21 17:48:41 +08:00
Mingfei
5240795154
edit according to comments
2013-06-21 17:38:23 +08:00
Matei Zaharia
1057fccf2a
Merge branch 'master' of github.com:mesos/spark
2013-06-20 18:46:59 +02:00
Matei Zaharia
ae7a5da6b3
Fix some dependency issues in SBT build (same will be needed for Maven):
...
- Exclude a version of ASM 3.x that comes from HBase
- Don't use a special ASF repo for HBase
- Update SLF4J version
- Add sbt-dependency-graph plugin so we can easily find dependency trees
2013-06-20 18:44:46 +02:00
Matei Zaharia
71030ba3eb
Merge pull request #654 from lyogavin/enhance_pipe
...
fix typo and coding style in #638
2013-06-19 15:21:03 -07:00
Matei Zaharia
7902baddc7
Update ASM to version 4.0
2013-06-19 13:34:30 +02:00