Ewen Cheslack-Postava
d17c142615
Force pseudo-tty allocation in spark-ec2 script.
...
ssh commands need the -t argument repeated twice if there is no local
tty, e.g. if the process running spark-ec2 uses nohup and the parent
process exits.
2013-12-16 08:09:37 -08:00
Ankur Dave
bc9f7eacb9
Enable stopping and starting a spot cluster
2013-11-11 17:50:31 -08:00
Haoyuan Li
6f455553c9
expose UI port only
2013-11-10 16:00:09 -08:00
Haoyuan Li
77cedf81c7
add tachyon module
2013-11-09 22:52:23 -08:00
Fabrizio (Misto) Milo
4b5d61f31f
update default github
2013-11-01 18:41:49 -07:00
Patrick Wendell
3745a1827f
Adding Shark 0.7.1 to EC2 scripts
2013-10-07 15:03:42 -07:00
Patrick Wendell
2484b84678
Bumping EC2 default version in master to 0.8.0.
2013-10-05 16:59:11 -07:00
Reynold Xin
76677b8fa1
Merge pull request #670 from jey/ec2-ssh-improvements
...
EC2 SSH improvements
2013-09-26 14:03:46 -07:00
Aaron Davidson
f589ce771a
Fix issue with spark_ec2 seeing empty security groups
...
Under unknown, but occasional, circumstances, reservation.groups is empty
despite reservation.instances each having groups. This means that the
spark_ec2 get_existing_clusters() method would fail to find any instances.
To fix it, we simply use the instances' groups as the source of truth.
Note that this is actually just a revival of PR #827 , now that the issue
has been reproduced.
2013-09-19 14:09:26 -07:00
Jey Kottalam
e86d1d4a52
Clarify error messages on SSH failure
2013-09-11 14:59:42 -07:00
Patrick Wendell
bddf135670
Change port from 3030 to 4040
2013-09-11 10:01:38 -07:00
Jey Kottalam
b98572c70a
Generate new SSH key for the cluster, make "--identity-file" optional
2013-09-06 14:51:47 -07:00
Jey Kottalam
6919a28d51
Construct shell commands as sequences for safety and composability
2013-09-06 14:28:26 -07:00
Patrick Wendell
c02585ea13
Make initial connection failure message less daunting.
...
Right now it seems like something has gone wrong when this message is printed out.
Instead, this is a normal condition. So I changed the message a bit.
2013-08-21 15:45:45 -07:00
Patrick Wendell
6be6b71c8c
Merge branch 'master' into ec2-updates
...
Conflicts:
ec2/spark_ec2.py
2013-08-21 15:34:31 -07:00
Patrick Wendell
51a1a0c602
Bump spark version
2013-08-20 22:14:52 -07:00
Patrick Wendell
07e5c8b695
Set default Hadoop version to 1
2013-08-20 15:49:52 -07:00
Matei Zaharia
793a722f8e
Allow some wiggle room in UISuite port test and in EC2 ports
2013-08-19 18:51:00 -07:00
Matei Zaharia
498a26189b
Small fixes to web UI:
...
- Use SPARK_PUBLIC_DNS environment variable if set (for EC2)
- Use a non-ephemeral port (3030 instead of 33000) by default
- Updated test to use non-ephemeral port too
2013-08-19 18:17:49 -07:00
Patrick Wendell
7e1b7d8a96
Adding web UI port
2013-08-08 14:17:11 -07:00
Patrick Wendell
1105766295
Point to new repository/branch
2013-08-01 15:43:41 -07:00
Patrick Wendell
56715587e4
Pointing to new EC2 repository
2013-08-01 12:27:43 -07:00
Patrick Wendell
5cc725a0e3
Merge branch 'master' into ec2-updates
...
Conflicts:
ec2/deploy.generic/root/mesos-ec2/ec2-variables.sh
2013-07-31 21:35:12 -07:00
Patrick Wendell
893aaff738
Don't install MapReduce for Hadoop1
2013-07-30 14:55:04 -07:00
Matei Zaharia
207548b67b
Open up Job UI ports (33000-33010) on EC2 clusters
2013-07-29 17:19:33 -07:00
Patrick Wendell
31b3c1df54
Small improvement in python script
2013-07-17 17:43:15 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Patrick Wendell
8dccee16af
Bug fix
2013-07-11 16:50:27 -07:00
Patrick Wendell
31c18a2528
Removing support for custom Shark version
2013-07-11 16:18:16 -07:00
Patrick Wendell
13809d363c
Small fix
2013-07-11 15:45:35 -07:00
Patrick Wendell
f1689185d6
Adding ability to custom deploy only Spark
2013-07-11 15:44:31 -07:00
Patrick Wendell
13e3dd98d8
Removing mesos support
2013-06-26 11:36:39 -07:00
Patrick Wendell
d5f74aa9b7
Cleanup of AMI tag
2013-06-13 10:30:59 -07:00
Patrick Wendell
eae7590d0d
Support for multiple hadoop versions
2013-06-12 18:39:01 -07:00
Patrick Wendell
bb328c8ff8
Re-working spark versions to use templates
2013-06-11 20:14:33 -07:00
Patrick Wendell
2124563a00
Remving support for old scripts
2013-06-11 17:52:10 -07:00
Patrick Wendell
8637c02191
Minor doc cleanup
2013-06-07 14:54:26 -07:00
Patrick Wendell
0f4c0e461f
Removing now defunct ami documentation
2013-06-07 14:39:58 -07:00
Patrick Wendell
d65d3dd8f0
Fixing syntax error
2013-06-07 14:16:01 -07:00
Patrick Wendell
2477736237
Adding spark shark modules
2013-06-07 11:55:46 -07:00
Patrick Wendell
db50379737
Decoupling spark version from ami
2013-06-07 11:54:16 -07:00
Patrick Wendell
a07bd286bb
Some changes in response to Josh
2013-05-18 16:00:32 -07:00
Patrick Wendell
e8a1d02956
Adding support for tag
2013-05-10 15:13:22 -07:00
Patrick Wendell
2197b2159a
Removing unnecessary parsing
2013-05-09 11:27:13 -07:00
Patrick Wendell
36c117a1f1
Slight change to AMI versioning
2013-05-08 22:50:53 -07:00
Patrick Wendell
f6c965a98b
Changing spark version and availability zone fix
2013-05-08 21:46:01 -07:00
Patrick Wendell
976e02491e
Resolve AMI region automatically
2013-05-08 21:46:01 -07:00
Patrick Wendell
f8213aab4b
Adding automatic resolve of AMI
2013-05-08 21:46:01 -07:00
Patrick Wendell
03b58378d0
Adding mapreduce module
2013-05-08 21:46:01 -07:00
Patrick Wendell
73f8cd237e
Adding extra instance types
2013-05-08 21:46:01 -07:00
