ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
1451 commits 6 branches 181 tags 397 MiB
Commit graph

1451 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia e7051767f7 Merge pull request #337 from pwendell/worker-liveness-ui 
SPARK-616: Logging dead workers in Web UI.
 2012-12-19 15:31:32 -08:00
Matei Zaharia 30b47794da Merge pull request #340 from tomdz/deb-packaging-tweaks 
Tweaked debian packaging to be a bit more in line with debian standards
 2012-12-19 12:07:03 -08:00
Thomas Dudziak 5488ac67c3 Tweaked debian packaging to be a bit more in line with debian standards 2012-12-19 10:20:43 -08:00
Matei Zaharia 1e6e154d6d Merge pull request #338 from tomdz/repl-pom-fix 
Fixed repl maven build
 2012-12-18 14:03:29 -08:00
Thomas Dudziak 4af6cad37a Fixed repl maven build to produce artifacts with the appropriate hadoop classifier and extracted repl fat-jar and debian packaging into a separate project to make Maven happy 2012-12-18 12:08:19 -08:00
Patrick Wendell bfac06e1f6 SPARK-616: Logging dead workers in Web UI. 
This patch keeps track of which workers have died and marks them
as such in the master web UI. It also handles workers which die and
re-register using different actor ID's.
 2012-12-17 23:09:05 -08:00
Matei Zaharia b82a6dd2c7 Merge pull request #332 from JoshRosen/spark-607 
Add try-finally to handle MapOutputTracker timeouts
 2012-12-14 11:41:16 -08:00
Matei Zaharia 1072f970cc Merge pull request #331 from woggling/deploy-exit-status 
Have standalone cluster report exit codes to clients
 2012-12-13 22:43:48 -08:00
Charles Reiss c528932a41 Code review cleanup. 2012-12-13 22:37:16 -08:00
Charles Reiss 0aad42b5e7 Have standalone cluster report exit codes to clients. Addresses SPARK-639. 2012-12-13 22:37:16 -08:00
Matei Zaharia d6d910471d Merge pull request #333 from rxin/master 
Fixed the broken Java unit test from SPARK-635.
 2012-12-13 22:31:53 -08:00
Reynold Xin f4a9e1b9be Fixed the broken Java unit test from SPARK-635. 2012-12-13 22:22:12 -08:00
Josh Rosen cf52d9cade Add try-finally to handle MapOutputTracker timeouts. 2012-12-13 21:53:30 -08:00
Matei Zaharia 05e225f988 Merge pull request #329 from woggling/executor-status-codes 
Executor exit status codes
 2012-12-13 20:14:10 -08:00
Charles Reiss b054d3b222 ExecutorLostReason -> ExecutorLossReason 2012-12-13 18:44:07 -08:00
Charles Reiss 24d7aa2d15 Extra whitespace in ExecutorExitCode 2012-12-13 18:39:23 -08:00
Matei Zaharia 012aa4e3d4 Merge pull request #330 from JoshRosen/spark-638 
Use spark-env.sh to configure standalone master
 2012-12-13 18:05:32 -08:00
Josh Rosen 1948f46093 Use spark-env.sh to configure standalone master. See SPARK-638. 
Also fixed a typo in the standalone mode documentation.
 2012-12-14 01:20:00 +00:00
Matei Zaharia dfcbb2ad97 Merge pull request #328 from rxin/spark-635 
SPARK-635: Pass a TaskContext object to compute() interface and use that to close Hadoop input stream.
 2012-12-13 16:46:54 -08:00
Reynold Xin 4f076e105e SPARK-635: Pass a TaskContext object to compute() interface and use 
that to close Hadoop input stream. Incorporated Matei's command.
 2012-12-13 16:41:15 -08:00
Charles Reiss 829206f1a7 Explain slaveLost calls made by StandaloneSchedulerBackend 2012-12-13 16:23:36 -08:00
Charles Reiss a4041dd87f Log duplicate slaveLost() calls in ClusterScheduler. 2012-12-13 16:23:36 -08:00
Charles Reiss fa9df4a45d Normalize executor exit statuses and report them to the user. 2012-12-13 16:23:31 -08:00
Reynold Xin eacb98e900 SPARK-635: Pass a TaskContext object to compute() interface and use that 
to close Hadoop input stream.
 2012-12-13 15:41:53 -08:00
Matei Zaharia 8fd374dfb2 Merge pull request #325 from tomdz/master 
Mark hadoop dependencies provided in all library artifacts
 2012-12-13 12:44:20 -08:00
Charles Reiss 1d8e2e6cff Call slaveLost on executor death for standalone clusters. 2012-12-12 21:15:34 -08:00
Matei Zaharia 391e5a194a Merge pull request #323 from admobius/master 
SPARK-626: Remove security groups properly
 2012-12-11 12:01:42 -08:00
Peter Sankauskas f97ce3ae14 SPARK-626: Making security group deletion optional, handling retried 
when deleting security groups fails, fixing bug when using all zones but
only 1 slave.
 2012-12-11 10:48:21 -08:00
Thomas Dudziak 02d64f9662 Mark hadoop dependencies provided in all library artifacts 2012-12-10 21:27:54 -08:00
Matei Zaharia 75b8d1f870 Merge pull request #324 from rxin/master 
Suppress shuffle block updates when a slave node comes back.
 2012-12-10 20:39:08 -08:00
Reynold Xin 21b271f5bd Suppress shuffle block updates when a slave node comes back. 2012-12-10 20:36:03 -08:00
Peter Sankauskas 9f964612a1 SPARK-626: Remove rules before removing security groups, with a pause in 
between so wait for AWS eventual consistency to catch up.
 2012-12-10 17:44:09 -08:00
Matei Zaharia c10b229992 Merge pull request #319 from mbautin/cdh4.1.2 
Bump CDH version for the Hadoop 2 profile to 4.1.2
 2012-12-10 15:21:01 -08:00
Matei Zaharia 1ae8978298 Merge pull request #320 from tomdz/master 
Shaded repl jar for hadoop1 profile needs to include hadoop classes
 2012-12-10 15:15:10 -08:00
Matei Zaharia 01c1f97e95 Make "run" script work with Maven builds 2012-12-10 15:13:16 -08:00
Matei Zaharia 597520ae20 Make sure the SSH key we copy to EC2 has permissions 600. 
SPARK-539 #resolve
 2012-12-10 15:12:06 -08:00
Thomas Dudziak c1d15ae3d5 Shaded repl jar for hadoop1 profile needs to include hadoop classes 2012-12-10 15:06:28 -08:00
Mikhail Bautin 450659079a Bump CDH version for the Hadoop 2 profile to 4.1.2 2012-12-10 11:27:20 -08:00
Matei Zaharia a1a2daa7ef Merge pull request #317 from woggling/block-manager-heartbeat 
Implement block manager heartbeat
 2012-12-10 11:03:55 -08:00
Matei Zaharia a9ea14d6e7 Merge pull request #318 from tomdz/master 
Minor tweaks to the debian build
 2012-12-10 10:59:41 -08:00
Matei Zaharia ccff0a089a Use the same output directories that SBT had in subprojects 
This will make it easier to make the "run" script work with a Maven build
 2012-12-10 10:58:56 -08:00
Thomas Dudziak 0e5b1f7981 Minor tweaks to the debian build 2012-12-10 10:30:30 -08:00
Charles Reiss b6b62d774f Decrease BlockManagerMaster logging verbosity 2012-12-10 00:31:55 -08:00
Charles Reiss 5d3e917d09 Use Akka scheduler for BlockManager heart beats. 
Adds required ActorSystem argument to BlockManager constructors.
 2012-12-10 00:31:50 -08:00
Charles Reiss b53dd28c90 Changed default block manager heartbeat interval to 5 s 2012-12-09 23:03:34 -08:00
Matei Zaharia beb440089e Merge pull request #310 from tomdz/master-mavenized 
Maven build setup
 2012-12-09 21:40:05 -08:00
Matei Zaharia e1d7cd2276 Search for a non-loopback address in Utils.getLocalIpAddress 2012-12-08 00:33:11 -08:00
Charles Reiss 714c8d32d5 Don't divide by milliseconds by 1000 more. 2012-12-06 18:38:34 -08:00
Charles Reiss 8f0819520c map -> foreach 2012-12-06 18:29:50 -08:00
Charles Reiss 7a033fd795 Make LocalSparkCluster use distinct IPs 2012-12-06 00:03:08 -08:00