ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/dev/ansible-for-test-node
History
shane knapp 2c94fbc71e initial commit for skeleton ansible for jenkins worker config 
### What changes were proposed in this pull request?
this is the skeleton of the ansible used to configure jenkins workers in the riselab/apache spark build system

### Why are the changes needed?
they are not needed, but will help the community understand how to build systems to test multiple versions of spark, as well as propose changes that i can integrate in to the "production" riselab repo.  since we're sunsetting jenkins by EOY 2021, this will potentially be useful for migrating the build system.

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
ansible-lint and much wailing and gnashing of teeth.

Closes #32178 from shaneknapp/initial-ansible-commit.

Lead-authored-by: shane knapp <incomplete@gmail.com>
Co-authored-by: shane <incomplete@gmail.com>
Signed-off-by: shane knapp <incomplete@gmail.com>
2021-06-30 10:05:27 -07:00
..
roles initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00
deploy-jenkins-worker.yml initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00
README.md initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00

README.md

jenkins-infra

This is a rough skeleton of the ansible used to deploy RISELab/Apache Spark Jenkins build workers on Ubuntu 20LTS.

WARNING: this will not work "directly out of the box" and will need to be tweaked to work on any ubuntu servers you might want to try this on.

deploy a new worker node

TL;DR:

all of the configs for the workers live in roles/common/... and roles/jenkins-worker...

prereqs:

  • fresh install of ubuntu 20
  • a service account w/sudo
  • python 3, ansible, ansible-playbook installed locally
  • add hostname(s) to the hosts file
  • add this to your ~/.ansible.cfg: [defaults] host_key_checking = False

fire ansible cannon!

ansible-playbook -u <service account> deploy-jenkins-worker.yml -i <ansible-style-hosts-file> -k -b -K

tips:

  • if you are installing more than a few workers, it's best to run the playbook on smaller (2-3) batches at a time. this way it's easier to track down errors, as ansible is very noisy.
  • when you encounter an error, you should comment out any previously-run plays and tasks. this saves time when debugging, and let's you easily track where you are in the process.
  • apt-get remove <application_name> and apt-get purge <package-name> are your friends