Fix issue with spark_ec2 seeing empty security groups
Under unknown, but occasional, circumstances, reservation.groups is empty despite reservation.instances each having groups. This means that the spark_ec2 get_existing_clusters() method would fail to find any instances. To fix it, we simply use the instances' groups as the source of truth. Note that this is actually just a revival of PR #827, now that the issue has been reproduced.
This commit is contained in:
parent
2aff7989ab
commit
f589ce771a
|
@ -364,12 +364,12 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True):
|
|||
slave_nodes = []
|
||||
for res in reservations:
|
||||
active = [i for i in res.instances if is_active(i)]
|
||||
if len(active) > 0:
|
||||
group_names = [g.name for g in res.groups]
|
||||
for inst in active:
|
||||
group_names = [g.name for g in inst.groups]
|
||||
if group_names == [cluster_name + "-master"]:
|
||||
master_nodes += res.instances
|
||||
master_nodes.append(inst)
|
||||
elif group_names == [cluster_name + "-slaves"]:
|
||||
slave_nodes += res.instances
|
||||
slave_nodes.append(inst)
|
||||
if any((master_nodes, slave_nodes)):
|
||||
print ("Found %d master(s), %d slaves" %
|
||||
(len(master_nodes), len(slave_nodes)))
|
||||
|
|
Loading…
Reference in a new issue