ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/dev
History
shane knapp 2c94fbc71e initial commit for skeleton ansible for jenkins worker config 
### What changes were proposed in this pull request?
this is the skeleton of the ansible used to configure jenkins workers in the riselab/apache spark build system

### Why are the changes needed?
they are not needed, but will help the community understand how to build systems to test multiple versions of spark, as well as propose changes that i can integrate in to the "production" riselab repo.  since we're sunsetting jenkins by EOY 2021, this will potentially be useful for migrating the build system.

### Does this PR introduce _any_ user-facing change?
no

### How was this patch tested?
ansible-lint and much wailing and gnashing of teeth.

Closes #32178 from shaneknapp/initial-ansible-commit.

Lead-authored-by: shane knapp <incomplete@gmail.com>
Co-authored-by: shane <incomplete@gmail.com>
Signed-off-by: shane knapp <incomplete@gmail.com>
2021-06-30 10:05:27 -07:00
..
ansible-for-test-node initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00
create-release [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts 2021-06-30 16:24:03 +09:00
deps [SPARK-35784][SS] Implementation for RocksDB instance 2021-06-29 17:46:45 -07:00
sparktestsupport Revert "[SPARK-35721][PYTHON] Path level discover for python unittests" 2021-06-29 12:08:09 -07:00
tests Spelling r common dev mlib external project streaming resource managers python 2020-11-27 10:22:45 -06:00
.gitignore [SPARK-23174][BUILD][PYTHON][FOLLOWUP] Add pycodestyle*.py to .gitignore file. 2018-01-31 00:51:00 +09:00
.rat-excludes initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00
.scalafmt.conf [SPARK-35255][BUILD] Automated formatting for Scala Code for Blank Lines 2021-04-30 11:45:58 +09:00
appveyor-guide.md Spelling r common dev mlib external project streaming resource managers python 2020-11-27 10:22:45 -06:00
appveyor-install-dependencies.ps1 [SPARK-33105][INFRA] Change default R arch from i386 to x64 and parametrize BINPREF 2020-10-10 13:48:26 +09:00
change-scala-version.sh [SPARK-34762][BUILD] Fix the build failure with Scala 2.13 which is related to commons-cli 2021-03-18 12:31:50 +09:00
check-license [MINOR][INFRA] Suppress warning in check-license 2020-11-23 10:38:40 +09:00
checkstyle-suppressions.xml [SPARK-29674][CORE] Update dropwizard metrics to 4.1.x for JDK 9+ 2019-11-03 15:13:06 -08:00
checkstyle.xml [SPARK-35609][BUILD] Add style rules to prohibit to use a Guava's API which is incompatible with newer versions 2021-06-03 21:52:41 +09:00
eslint.json [SPARK-35175][BUILD] Add linter for JavaScript source files 2021-05-07 21:55:08 +09:00
github_jira_sync.py Spelling r common dev mlib external project streaming resource managers python 2020-11-27 10:22:45 -06:00
lint-java [SPARK-23063][K8S] K8s changes for publishing scripts (and a couple of other misses) 2018-01-13 21:34:28 -08:00
lint-js [SPARK-35175][BUILD] Add linter for JavaScript source files 2021-05-07 21:55:08 +09:00
lint-python [SPARK-35499][PYTHON] Apply black to pandas API on Spark codes 2021-06-06 17:30:07 -07:00
lint-r [SPARK-29932][R][TESTS] lint-r should do non-zero exit in case of errors 2019-11-17 10:09:46 -08:00
lint-r.R [MINOR][R] small tidying of sh scripts for R 2020-04-30 16:58:05 -07:00
lint-scala [SPARK-27158][BUILD] dev/mima and dev/scalastyle support dynamic profiles 2019-03-15 08:20:42 +09:00
make-distribution.sh [SPARK-31041][BUILD] Show Maven errors from within make-distribution.sh 2020-03-11 08:22:02 -05:00
merge_spark_pr.py [MINOR] Fix usage print to guide pip3 to install jira-python library 2020-09-03 01:10:59 +09:00
mima [SPARK-33510][BUILD] Update SBT to 1.4.4 2020-11-22 22:56:59 -08:00
package-lock.json [SPARK-35175][BUILD] Add linter for JavaScript source files 2021-05-07 21:55:08 +09:00
package.json [SPARK-35175][BUILD] Add linter for JavaScript source files 2021-05-07 21:55:08 +09:00
pip-sanity-check.py [SPARK-32319][PYSPARK] Disallow the use of unused imports 2020-08-08 08:51:57 -07:00
README.md Merge pull request #565 from pwendell/dev-scripts. Closes #565. 2014-02-08 23:13:34 -08:00
reformat-python [SPARK-35499][PYTHON] Apply black to pandas API on Spark codes 2021-06-06 17:30:07 -07:00
requirements.txt [SPARK-35499][PYTHON] Apply black to pandas API on Spark codes 2021-06-06 17:30:07 -07:00
run-pip-tests [SPARK-35393][PYTHON][INFRA][TESTS] Recover pip packaging test in Github Actions 2021-05-13 10:35:56 -07:00
run-tests [SPARK-29672][PYSPARK] update spark testing framework to use python3 2019-11-14 10:18:55 -08:00
run-tests-jenkins [SPARK-33535][INFRA][TESTS] Export LANG to en_US.UTF-8 in run-tests-jenkins script 2020-11-24 09:50:10 -08:00
run-tests-jenkins.py Spelling r common dev mlib external project streaming resource managers python 2020-11-27 10:22:45 -06:00
run-tests.py [SPARK-35483][FOLLOWUP][TESTS] Update run-tests.py doctest 2021-06-28 23:14:47 -07:00
sbt-checkstyle [SPARK-27158][BUILD] dev/mima and dev/scalastyle support dynamic profiles 2019-03-15 08:20:42 +09:00
scalafmt [SPARK-30570][BUILD] Update scalafmt plugin to 1.0.3 with onlyChangedFiles feature 2020-01-23 12:44:43 -08:00
scalastyle Revert "[SPARK-30534][INFRA] Use mvn in dev/scalastyle" 2020-01-21 18:23:03 +09:00
test-dependencies.sh [SPARK-20202][BUILD][SQL] Remove references to org.spark-project.hive (Hive 1.2.1) 2020-10-05 15:29:56 -07:00
tox.ini initial commit for skeleton ansible for jenkins worker config 2021-06-30 10:05:27 -07:00

README.md

Spark Developer Scripts

This directory contains scripts useful to developers when packaging, testing, or committing to Spark.

Many of these scripts require Apache credentials to work correctly.