Matei Zaharia
f9fa2add5c
Merge pull request #530 from mbautin/master-update-log4j-and-make-compile-in-IntelliJ
...
Add a log4j compile dependency to fix build in IntelliJ
2013-03-15 15:12:43 -07:00
Mikhail Bautin
7fd2708eda
Add a log4j compile dependency to fix build in IntelliJ
...
Also rename parent project to spark-parent (otherwise it shows up as
"parent" in IntelliJ, which is very confusing).
2013-03-15 11:41:51 -07:00
Matei Zaharia
4032beba49
Merge pull request #521 from stephenh/earlyclose
...
Close the reader in HadoopRDD as soon as iteration end.
2013-03-13 19:29:46 -07:00
Matei Zaharia
3c97276a4a
Merge pull request #524 from andyk/master
...
Fix broken link to YARN documentation
2013-03-13 19:25:08 -07:00
Andy Konwinski
c9097628fc
Fix broken link to YARN documentation.
2013-03-13 14:51:13 -07:00
Stephen Haberman
e7f1a69c6b
Add a test for NextIterator.
2013-03-13 10:46:33 -05:00
Stephen Haberman
1a175d13b9
Add NextIterator.closeIfNeeded.
2013-03-13 10:17:39 -05:00
Matei Zaharia
2d477fd36f
Merge pull request #523 from andyk/master
...
Fix broken link in Quick Start
2013-03-13 06:49:16 -07:00
Andy Konwinski
cf73fbd305
Fix another broken link in quick start.
2013-03-13 02:23:44 -07:00
Andy Konwinski
b63109763b
Fix broken link in Quick Start.
2013-03-13 02:02:34 -07:00
Matei Zaharia
00c4d238dd
Merge pull request #518 from woggling/long-bm-sizes
...
Send block sizes as longs in BlockManager updates
2013-03-12 22:19:00 -07:00
Stephen Haberman
8f00d23598
Remove NextIterator.close default implementation.
2013-03-12 12:30:10 -05:00
Stephen Haberman
0cf320485d
Forgot equals.
2013-03-12 00:05:35 -05:00
Stephen Haberman
9e68f48625
More quickly call close in HadoopRDD.
...
This also refactors out the common "gotNext" iterator pattern into
a shared utility class.
2013-03-11 23:59:17 -05:00
Charles Reiss
769d399674
Send block sizes as longs.
2013-03-11 14:17:05 -07:00
Matei Zaharia
cbf8f0d4dd
Merge pull request #513 from MLnick/bagel-caching
...
Adds choice of persistence level to Bagel.
2013-03-11 00:23:57 -07:00
Nick Pentreath
8dd943fc3e
Fix doc style
2013-03-11 09:14:00 +02:00
Matei Zaharia
2e1bbc4e7e
Merge remote-tracking branch 'woggling/dag-sched-driver-port'
...
Conflicts:
core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
2013-03-10 16:52:54 -07:00
Matei Zaharia
d4e29ea878
Update kryo-serializers version in pom.xml to match previous commit
2013-03-10 15:49:11 -07:00
Matei Zaharia
91a9d093bd
Merge pull request #512 from patelh/fix-kryo-serializer
...
Fix reference bug in Kryo serializer, add test, update version
2013-03-10 15:48:23 -07:00
Matei Zaharia
557cfd0f4d
Merge pull request #515 from woggling/deploy-app-death
...
Notify standalone deploy client of application death.
2013-03-10 15:44:57 -07:00
Matei Zaharia
a59cc6060f
Merge remote-tracking branch 'stephenh/nomocks'
...
Conflicts:
core/src/main/scala/spark/storage/BlockManagerMaster.scala
core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
2013-03-10 13:39:10 -07:00
Charles Reiss
b0983c5762
Notify standalone deploy client of application death.
...
Usually, this isn't necessary since the application will be removed
as a result of the deploy client disconnecting, but occassionally, the
standalone deploy master removes an application otherwise.
Also mark applications as FAILED instead of FINISHED when they are
killed as a result of their executors failing too many times.
2013-03-09 11:29:45 -08:00
Charles Reiss
d0216cb38b
Prevent DAGSchedulerSuite from corrupting driver.port.
...
Use the LocalSparkContext abstraction to properly manage clearing
spark.driver.port.
2013-03-09 10:49:02 -08:00
Nick Pentreath
d35c5a5176
Adding test for non-default persistence level
2013-03-09 12:52:16 +02:00
Nick Pentreath
1e981d8b26
Added choice of persitance level to Bagel. Also added documentation.
2013-03-09 12:40:48 +02:00
Hiral Patel
664e5fd24b
Fix reference bug in Kryo serializer, add test, update version
2013-03-07 22:16:11 -08:00
Matei Zaharia
9f0dc829cb
Fix TaskMetrics not being serializable
2013-03-04 12:08:31 -08:00
Matei Zaharia
04fb81ffe5
Merge pull request #506 from rxin/spark-706
...
Fixed SPARK-706: Failures in block manager put leads to read task hanging.
2013-03-03 17:20:07 -08:00
Matei Zaharia
6cf4be4b39
Merge pull request #462 from squito/stageInfo
...
Track assorted metrics for each task, report summaries to user at stage completion
2013-03-03 17:16:22 -08:00
Imran Rashid
0bd1d00c2a
minor cleanup based on feedback in review request
2013-03-03 16:46:45 -08:00
Imran Rashid
f1006b99ff
change CleanupIterator to CompletionIterator
2013-03-03 16:39:05 -08:00
Imran Rashid
8fef5b9c5f
refactoring of TaskMetrics
2013-03-03 16:34:04 -08:00
Imran Rashid
d36abdb053
Merge branch 'master' into stageInfo
2013-03-03 15:20:46 -08:00
Matei Zaharia
6bfc7cad6b
Merge pull request #504 from mosharaf/master
...
Worker address was getting removed when removing an app.
2013-03-02 22:14:49 -08:00
Matei Zaharia
94b3db136d
Merge pull request #508 from markhamstra/TestServerInUse
...
Avoid bind failure in InputStreamsSuite
2013-03-02 22:13:52 -08:00
Mark Hamstra
b409073102
Instead of failing to bind to a fixed, already-in-use port, let the OS choose an available port for TestServer.
2013-03-01 15:05:07 -08:00
Matei Zaharia
25c71d3e12
Merge pull request #507 from markhamstra/poms271
...
bump version to 0.7.1-SNAPSHOT in the subproject poms
2013-03-01 08:00:18 -08:00
Mark Hamstra
8b06b359da
bump version to 0.7.1-SNAPSHOT in the subproject poms to keep the maven build building.
2013-02-28 23:34:34 -08:00
Reynold Xin
44134e12bb
Fixed SPARK-706: Failures in block manager put leads to read task
...
hanging.
2013-02-28 15:14:59 -08:00
Mosharaf Chowdhury
4ab387bcdb
Fixed master datastructure updates after removing an application; and a typo.
2013-02-27 13:52:44 -08:00
Matei Zaharia
db9b90fdbd
Change version to 0.7.1-SNAPSHOT for development branch
2013-02-27 09:15:26 -08:00
Matei Zaharia
4f840f4e54
Added commented-out Google analytics code for website docs
2013-02-27 09:14:11 -08:00
Matei Zaharia
baa30fcd99
Use new Spark EC2 scripts by default
2013-02-26 23:38:50 -08:00
Matei Zaharia
fadeb1ddea
More doc tweaks
2013-02-26 23:20:49 -08:00
Matei Zaharia
22334eafd9
Some tweaks to docs
2013-02-26 22:52:38 -08:00
Matei Zaharia
434a1ce773
Small hack to work around multiple JARs being built by sbt package
2013-02-26 12:24:18 -08:00
Matei Zaharia
ece3edfffa
Fix a problem with no hosts being counted as alive in the first job
2013-02-26 12:11:03 -08:00
Matei Zaharia
73697e2891
Fix overly large thread names in PySpark
2013-02-26 12:07:59 -08:00
Stephen Haberman
db957e5bd7
Fix MapOutputTrackerSuite.
2013-02-26 01:38:50 -06:00