From 754f820035477f9538de6417cf68241b98ba82c6 Mon Sep 17 00:00:00 2001 From: Sean Owen Date: Sat, 30 Mar 2019 19:49:45 -0500 Subject: [PATCH] [SPARK-26918][DOCS] All .md should have ASF license header ## What changes were proposed in this pull request? Add AL2 license to metadata of all .md files. This seemed to be the tidiest way as it will get ignored by .md renderers and other tools. Attempts to write them as markdown comments revealed that there is no such standard thing. ## How was this patch tested? Doc build Closes #24243 from srowen/SPARK-26918. Authored-by: Sean Owen Signed-off-by: Sean Owen --- R/CRAN_RELEASE.md | 18 +++++++++++++++ R/DOCUMENTATION.md | 18 +++++++++++++++ R/WINDOWS.md | 18 +++++++++++++++ .../apache/spark/deploy/security/README.md | 18 +++++++++++++++ dev/appveyor-guide.md | 18 +++++++++++++++ docs/README.md | 18 +++++++++++++++ docs/api.md | 15 +++++++++++++ docs/building-spark.md | 15 +++++++++++++ docs/cloud-integration.md | 22 ++++++++++--------- docs/cluster-overview.md | 15 +++++++++++++ docs/configuration.md | 15 +++++++++++++ docs/contributing-to-spark.md | 15 +++++++++++++ docs/graphx-programming-guide.md | 15 +++++++++++++ docs/hadoop-provided.md | 15 +++++++++++++ docs/hardware-provisioning.md | 15 +++++++++++++ docs/index.md | 15 +++++++++++++ docs/job-scheduling.md | 15 +++++++++++++ docs/ml-advanced.md | 15 +++++++++++++ docs/ml-ann.md | 15 +++++++++++++ docs/ml-classification-regression.md | 15 +++++++++++++ docs/ml-clustering.md | 15 +++++++++++++ docs/ml-collaborative-filtering.md | 15 +++++++++++++ docs/ml-datasource.md | 15 +++++++++++++ docs/ml-decision-tree.md | 15 +++++++++++++ docs/ml-ensembles.md | 15 +++++++++++++ docs/ml-features.md | 15 +++++++++++++ docs/ml-frequent-pattern-mining.md | 15 +++++++++++++ docs/ml-guide.md | 15 +++++++++++++ docs/ml-linear-methods.md | 15 +++++++++++++ docs/ml-migration-guides.md | 15 +++++++++++++ docs/ml-pipeline.md | 15 +++++++++++++ docs/ml-statistics.md | 15 +++++++++++++ docs/ml-survival-regression.md | 15 +++++++++++++ docs/ml-tuning.md | 15 +++++++++++++ docs/mllib-classification-regression.md | 15 +++++++++++++ docs/mllib-clustering.md | 15 +++++++++++++ docs/mllib-collaborative-filtering.md | 15 +++++++++++++ docs/mllib-data-types.md | 15 +++++++++++++ docs/mllib-decision-tree.md | 15 +++++++++++++ docs/mllib-dimensionality-reduction.md | 15 +++++++++++++ docs/mllib-ensembles.md | 15 +++++++++++++ docs/mllib-evaluation-metrics.md | 15 +++++++++++++ docs/mllib-feature-extraction.md | 15 +++++++++++++ docs/mllib-frequent-pattern-mining.md | 15 +++++++++++++ docs/mllib-guide.md | 15 +++++++++++++ docs/mllib-isotonic-regression.md | 15 +++++++++++++ docs/mllib-linear-methods.md | 15 +++++++++++++ docs/mllib-migration-guides.md | 15 +++++++++++++ docs/mllib-naive-bayes.md | 15 +++++++++++++ docs/mllib-optimization.md | 15 +++++++++++++ docs/mllib-pmml-model-export.md | 15 +++++++++++++ docs/mllib-statistics.md | 15 +++++++++++++ docs/monitoring.md | 15 +++++++++++++ docs/programming-guide.md | 15 +++++++++++++ docs/quick-start.md | 15 +++++++++++++ docs/rdd-programming-guide.md | 15 +++++++++++++ docs/running-on-kubernetes.md | 15 +++++++++++++ docs/running-on-mesos.md | 15 +++++++++++++ docs/running-on-yarn.md | 15 +++++++++++++ docs/security.md | 15 +++++++++++++ docs/spark-standalone.md | 15 +++++++++++++ docs/sparkr.md | 15 +++++++++++++ docs/sql-data-sources-avro.md | 15 +++++++++++++ docs/sql-data-sources-hive-tables.md | 15 +++++++++++++ docs/sql-data-sources-jdbc.md | 15 +++++++++++++ docs/sql-data-sources-json.md | 15 +++++++++++++ docs/sql-data-sources-load-save-functions.md | 15 +++++++++++++ docs/sql-data-sources-orc.md | 15 +++++++++++++ docs/sql-data-sources-parquet.md | 15 +++++++++++++ docs/sql-data-sources-troubleshooting.md | 15 +++++++++++++ docs/sql-data-sources.md | 15 +++++++++++++ docs/sql-distributed-sql-engine.md | 15 +++++++++++++ docs/sql-getting-started.md | 15 +++++++++++++ docs/sql-keywords.md | 15 +++++++++++++ .../sql-migration-guide-hive-compatibility.md | 15 +++++++++++++ docs/sql-migration-guide-upgrade.md | 15 +++++++++++++ docs/sql-migration-guide.md | 15 +++++++++++++ docs/sql-performance-tuning.md | 15 +++++++++++++ docs/sql-programming-guide.md | 15 +++++++++++++ docs/sql-pyspark-pandas-with-arrow.md | 15 +++++++++++++ docs/sql-reference.md | 15 +++++++++++++ docs/storage-openstack-swift.md | 15 +++++++++++++ docs/streaming-custom-receivers.md | 15 +++++++++++++ docs/streaming-kafka-0-10-integration.md | 15 +++++++++++++ docs/streaming-kafka-integration.md | 15 +++++++++++++ docs/streaming-kinesis-integration.md | 15 +++++++++++++ docs/streaming-programming-guide.md | 15 +++++++++++++ .../structured-streaming-kafka-integration.md | 15 +++++++++++++ .../structured-streaming-programming-guide.md | 15 +++++++++++++ docs/submitting-applications.md | 15 +++++++++++++ docs/tuning.md | 15 +++++++++++++ 91 files changed, 1380 insertions(+), 10 deletions(-) diff --git a/R/CRAN_RELEASE.md b/R/CRAN_RELEASE.md index d6084c7a7c..4d9b6416c0 100644 --- a/R/CRAN_RELEASE.md +++ b/R/CRAN_RELEASE.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + # SparkR CRAN Release To release SparkR as a package to CRAN, we would use the `devtools` package. Please work with the diff --git a/R/DOCUMENTATION.md b/R/DOCUMENTATION.md index 7314a1fccc..fd0c764418 100644 --- a/R/DOCUMENTATION.md +++ b/R/DOCUMENTATION.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + # SparkR Documentation SparkR documentation is generated by using in-source comments and annotated by using diff --git a/R/WINDOWS.md b/R/WINDOWS.md index 33a4c850cf..92442f9b65 100644 --- a/R/WINDOWS.md +++ b/R/WINDOWS.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + ## Building SparkR on Windows To build SparkR on Windows, the following steps are required diff --git a/core/src/main/scala/org/apache/spark/deploy/security/README.md b/core/src/main/scala/org/apache/spark/deploy/security/README.md index c3ef60a231..0d98ce4391 100644 --- a/core/src/main/scala/org/apache/spark/deploy/security/README.md +++ b/core/src/main/scala/org/apache/spark/deploy/security/README.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + # Delegation Token Handling In Spark This document aims to explain and demystify delegation tokens as they are used by Spark, since diff --git a/dev/appveyor-guide.md b/dev/appveyor-guide.md index a842f39b30..a8c0c1ef23 100644 --- a/dev/appveyor-guide.md +++ b/dev/appveyor-guide.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + # AppVeyor Guides Currently, SparkR on Windows is being tested with [AppVeyor](https://ci.appveyor.com). This page describes how to set up AppVeyor with Spark, how to run the build, check the status and stop the build via this tool. There is the documentation for AppVeyor [here](https://www.appveyor.com/docs). Please refer this for full details. diff --git a/docs/README.md b/docs/README.md index fb67c4b358..5b90f98cf8 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,3 +1,21 @@ +--- +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. +--- + Welcome to the Spark documentation! This readme will walk you through navigating and building the Spark documentation, which is included diff --git a/docs/api.md b/docs/api.md index 70484f02de..241d552d14 100644 --- a/docs/api.md +++ b/docs/api.md @@ -1,6 +1,21 @@ --- layout: global title: Spark API Documentation +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Here you can read API docs for Spark and its submodules. diff --git a/docs/building-spark.md b/docs/building-spark.md index ef2f2bf231..ab80363819 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -2,6 +2,21 @@ layout: global title: Building Spark redirect_from: "building-with-maven.html" +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/cloud-integration.md b/docs/cloud-integration.md index a014d6fe0b..b64ffe55d8 100644 --- a/docs/cloud-integration.md +++ b/docs/cloud-integration.md @@ -3,20 +3,22 @@ layout: global displayTitle: Integration with Cloud Infrastructures title: Integration with Cloud Infrastructures description: Introduction to cloud storage support in Apache Spark SPARK_VERSION_SHORT ---- - + limitations under the License. +--- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/cluster-overview.md b/docs/cluster-overview.md index 1f0822f7a3..cb6d3a5c1f 100644 --- a/docs/cluster-overview.md +++ b/docs/cluster-overview.md @@ -1,6 +1,21 @@ --- layout: global title: Cluster Mode Overview +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- This document gives a short overview of how Spark runs on clusters, to make it easier to understand diff --git a/docs/configuration.md b/docs/configuration.md index 006d8395b5..3289954865 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -2,6 +2,21 @@ layout: global displayTitle: Spark Configuration title: Configuration +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/contributing-to-spark.md b/docs/contributing-to-spark.md index ede5584a0c..1e4cf4c0c1 100644 --- a/docs/contributing-to-spark.md +++ b/docs/contributing-to-spark.md @@ -1,6 +1,21 @@ --- layout: global title: Contributing to Spark +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The Spark team welcomes all forms of contributions, including bug reports, documentation or patches. diff --git a/docs/graphx-programming-guide.md b/docs/graphx-programming-guide.md index 3badb0a479..c96f7aaba8 100644 --- a/docs/graphx-programming-guide.md +++ b/docs/graphx-programming-guide.md @@ -3,6 +3,21 @@ layout: global displayTitle: GraphX Programming Guide title: GraphX description: GraphX graph processing library guide for Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/hadoop-provided.md b/docs/hadoop-provided.md index bbd26b343e..37cdaa6150 100644 --- a/docs/hadoop-provided.md +++ b/docs/hadoop-provided.md @@ -2,6 +2,21 @@ layout: global displayTitle: Using Spark's "Hadoop Free" Build title: Using Spark's "Hadoop Free" Build +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Spark uses Hadoop client libraries for HDFS and YARN. Starting in version Spark 1.4, the project packages "Hadoop free" builds that lets you more easily connect a single Spark binary to any Hadoop version. To use these builds, you need to modify `SPARK_DIST_CLASSPATH` to include Hadoop's package jars. The most convenient place to do this is by adding an entry in `conf/spark-env.sh`. diff --git a/docs/hardware-provisioning.md b/docs/hardware-provisioning.md index 29876a51b2..dab65e29b1 100644 --- a/docs/hardware-provisioning.md +++ b/docs/hardware-provisioning.md @@ -1,6 +1,21 @@ --- layout: global title: Hardware Provisioning +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- A common question received by Spark developers is how to configure hardware for it. While the right diff --git a/docs/index.md b/docs/index.md index 59cd3f6faa..5adbebdb74 100644 --- a/docs/index.md +++ b/docs/index.md @@ -3,6 +3,21 @@ layout: global displayTitle: Spark Overview title: Overview description: Apache Spark SPARK_VERSION_SHORT documentation homepage +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Apache Spark is a fast and general-purpose cluster computing system. diff --git a/docs/job-scheduling.md b/docs/job-scheduling.md index 2316f17567..3e70c59d89 100644 --- a/docs/job-scheduling.md +++ b/docs/job-scheduling.md @@ -1,6 +1,21 @@ --- layout: global title: Job Scheduling +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/ml-advanced.md b/docs/ml-advanced.md index 375957e92c..8e8c701da4 100644 --- a/docs/ml-advanced.md +++ b/docs/ml-advanced.md @@ -2,6 +2,21 @@ layout: global title: Advanced topics displayTitle: Advanced topics +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/ml-ann.md b/docs/ml-ann.md index 7c460c4af6..df5127d501 100644 --- a/docs/ml-ann.md +++ b/docs/ml-ann.md @@ -2,6 +2,21 @@ layout: global title: Multilayer perceptron classifier displayTitle: Multilayer perceptron classifier +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- > This section has been moved into the diff --git a/docs/ml-classification-regression.md b/docs/ml-classification-regression.md index 42912a2e2b..b83b4ba08a 100644 --- a/docs/ml-classification-regression.md +++ b/docs/ml-classification-regression.md @@ -2,6 +2,21 @@ layout: global title: Classification and regression displayTitle: Classification and regression +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- diff --git a/docs/ml-clustering.md b/docs/ml-clustering.md index 21e38cad15..2775d0421c 100644 --- a/docs/ml-clustering.md +++ b/docs/ml-clustering.md @@ -2,6 +2,21 @@ layout: global title: Clustering displayTitle: Clustering +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- This page describes clustering algorithms in MLlib. diff --git a/docs/ml-collaborative-filtering.md b/docs/ml-collaborative-filtering.md index 58646642bf..e6e596bed1 100644 --- a/docs/ml-collaborative-filtering.md +++ b/docs/ml-collaborative-filtering.md @@ -2,6 +2,21 @@ layout: global title: Collaborative Filtering displayTitle: Collaborative Filtering +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/ml-datasource.md b/docs/ml-datasource.md index 35afaef5ad..71bec9c798 100644 --- a/docs/ml-datasource.md +++ b/docs/ml-datasource.md @@ -2,6 +2,21 @@ layout: global title: Data sources displayTitle: Data sources +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- In this section, we introduce how to use data source in ML to load data. diff --git a/docs/ml-decision-tree.md b/docs/ml-decision-tree.md index 5e1eeb95e4..43c1fcafd2 100644 --- a/docs/ml-decision-tree.md +++ b/docs/ml-decision-tree.md @@ -2,6 +2,21 @@ layout: global title: Decision trees displayTitle: Decision trees +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- > This section has been moved into the diff --git a/docs/ml-ensembles.md b/docs/ml-ensembles.md index 97f1bdc803..bab04a6780 100644 --- a/docs/ml-ensembles.md +++ b/docs/ml-ensembles.md @@ -2,6 +2,21 @@ layout: global title: Tree ensemble methods displayTitle: Tree ensemble methods +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- > This section has been moved into the diff --git a/docs/ml-features.md b/docs/ml-features.md index 33373e0946..799b83504c 100644 --- a/docs/ml-features.md +++ b/docs/ml-features.md @@ -2,6 +2,21 @@ layout: global title: Extracting, transforming and selecting features displayTitle: Extracting, transforming and selecting features +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- This section covers algorithms for working with features, roughly divided into these groups: diff --git a/docs/ml-frequent-pattern-mining.md b/docs/ml-frequent-pattern-mining.md index f613664271..a243188603 100644 --- a/docs/ml-frequent-pattern-mining.md +++ b/docs/ml-frequent-pattern-mining.md @@ -2,6 +2,21 @@ layout: global title: Frequent Pattern Mining displayTitle: Frequent Pattern Mining +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Mining frequent items, itemsets, subsequences, or other substructures is usually among the diff --git a/docs/ml-guide.md b/docs/ml-guide.md index cffe41940e..4661d6cd87 100644 --- a/docs/ml-guide.md +++ b/docs/ml-guide.md @@ -2,6 +2,21 @@ layout: global title: "MLlib: Main Guide" displayTitle: "Machine Learning Library (MLlib) Guide" +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- MLlib is Spark's machine learning (ML) library. diff --git a/docs/ml-linear-methods.md b/docs/ml-linear-methods.md index eb39173505..047a6c79ff 100644 --- a/docs/ml-linear-methods.md +++ b/docs/ml-linear-methods.md @@ -2,6 +2,21 @@ layout: global title: Linear methods displayTitle: Linear methods +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- > This section has been moved into the diff --git a/docs/ml-migration-guides.md b/docs/ml-migration-guides.md index 2047065f71..99edd9bd69 100644 --- a/docs/ml-migration-guides.md +++ b/docs/ml-migration-guides.md @@ -3,6 +3,21 @@ layout: global title: Old Migration Guides - MLlib displayTitle: Old Migration Guides - MLlib description: MLlib migration guides from before Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The migration guide for the current Spark version is kept on the [MLlib Guide main page](ml-guide.html#migration-guide). diff --git a/docs/ml-pipeline.md b/docs/ml-pipeline.md index 0c9c998f63..993a428ab5 100644 --- a/docs/ml-pipeline.md +++ b/docs/ml-pipeline.md @@ -2,6 +2,21 @@ layout: global title: ML Pipelines displayTitle: ML Pipelines +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- `\[ diff --git a/docs/ml-statistics.md b/docs/ml-statistics.md index 6c82b3bb94..c404b62811 100644 --- a/docs/ml-statistics.md +++ b/docs/ml-statistics.md @@ -2,6 +2,21 @@ layout: global title: Basic Statistics displayTitle: Basic Statistics +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- diff --git a/docs/ml-survival-regression.md b/docs/ml-survival-regression.md index efa3c21c7c..e3bb133482 100644 --- a/docs/ml-survival-regression.md +++ b/docs/ml-survival-regression.md @@ -2,6 +2,21 @@ layout: global title: Survival Regression displayTitle: Survival Regression +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- > This section has been moved into the diff --git a/docs/ml-tuning.md b/docs/ml-tuning.md index 028bfec465..0717cce538 100644 --- a/docs/ml-tuning.md +++ b/docs/ml-tuning.md @@ -2,6 +2,21 @@ layout: global title: "ML Tuning" displayTitle: "ML Tuning: model selection and hyperparameter tuning" +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- `\[ diff --git a/docs/mllib-classification-regression.md b/docs/mllib-classification-regression.md index a7b90de093..d99c7ff053 100644 --- a/docs/mllib-classification-regression.md +++ b/docs/mllib-classification-regression.md @@ -2,6 +2,21 @@ layout: global title: Classification and Regression - RDD-based API displayTitle: Classification and Regression - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The `spark.mllib` package supports various methods for diff --git a/docs/mllib-clustering.md b/docs/mllib-clustering.md index dc6b095f5d..18bab115a2 100644 --- a/docs/mllib-clustering.md +++ b/docs/mllib-clustering.md @@ -2,6 +2,21 @@ layout: global title: Clustering - RDD-based API displayTitle: Clustering - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- [Clustering](https://en.wikipedia.org/wiki/Cluster_analysis) is an unsupervised learning problem whereby we aim to group subsets diff --git a/docs/mllib-collaborative-filtering.md b/docs/mllib-collaborative-filtering.md index aeebb26bb4..21546a6326 100644 --- a/docs/mllib-collaborative-filtering.md +++ b/docs/mllib-collaborative-filtering.md @@ -2,6 +2,21 @@ layout: global title: Collaborative Filtering - RDD-based API displayTitle: Collaborative Filtering - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-data-types.md b/docs/mllib-data-types.md index eca101132d..cdac46284b 100644 --- a/docs/mllib-data-types.md +++ b/docs/mllib-data-types.md @@ -2,6 +2,21 @@ layout: global title: Data Types - RDD-based API displayTitle: Data Types - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-decision-tree.md b/docs/mllib-decision-tree.md index 281755f4ce..045da74423 100644 --- a/docs/mllib-decision-tree.md +++ b/docs/mllib-decision-tree.md @@ -2,6 +2,21 @@ layout: global title: Decision Trees - RDD-based API displayTitle: Decision Trees - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-dimensionality-reduction.md b/docs/mllib-dimensionality-reduction.md index 4e6b453094..5eb36b4228 100644 --- a/docs/mllib-dimensionality-reduction.md +++ b/docs/mllib-dimensionality-reduction.md @@ -2,6 +2,21 @@ layout: global title: Dimensionality Reduction - RDD-based API displayTitle: Dimensionality Reduction - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-ensembles.md b/docs/mllib-ensembles.md index e1984b6c8d..6149f45821 100644 --- a/docs/mllib-ensembles.md +++ b/docs/mllib-ensembles.md @@ -2,6 +2,21 @@ layout: global title: Ensembles - RDD-based API displayTitle: Ensembles - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-evaluation-metrics.md b/docs/mllib-evaluation-metrics.md index 896d95bd48..f931fa32ea 100644 --- a/docs/mllib-evaluation-metrics.md +++ b/docs/mllib-evaluation-metrics.md @@ -2,6 +2,21 @@ layout: global title: Evaluation Metrics - RDD-based API displayTitle: Evaluation Metrics - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-feature-extraction.md b/docs/mllib-feature-extraction.md index bb29f65c03..b7f8ae9d07 100644 --- a/docs/mllib-feature-extraction.md +++ b/docs/mllib-feature-extraction.md @@ -2,6 +2,21 @@ layout: global title: Feature Extraction and Transformation - RDD-based API displayTitle: Feature Extraction and Transformation - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-frequent-pattern-mining.md b/docs/mllib-frequent-pattern-mining.md index 8e4505756b..8bc93ac2e8 100644 --- a/docs/mllib-frequent-pattern-mining.md +++ b/docs/mllib-frequent-pattern-mining.md @@ -2,6 +2,21 @@ layout: global title: Frequent Pattern Mining - RDD-based API displayTitle: Frequent Pattern Mining - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Mining frequent items, itemsets, subsequences, or other substructures is usually among the diff --git a/docs/mllib-guide.md b/docs/mllib-guide.md index 30112c72c9..dbb74407d0 100644 --- a/docs/mllib-guide.md +++ b/docs/mllib-guide.md @@ -2,6 +2,21 @@ layout: global title: "MLlib: RDD-based API" displayTitle: "MLlib: RDD-based API" +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- This page documents sections of the MLlib guide for the RDD-based API (the `spark.mllib` package). diff --git a/docs/mllib-isotonic-regression.md b/docs/mllib-isotonic-regression.md index 9964fce327..d9cc775547 100644 --- a/docs/mllib-isotonic-regression.md +++ b/docs/mllib-isotonic-regression.md @@ -2,6 +2,21 @@ layout: global title: Isotonic regression - RDD-based API displayTitle: Regression - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- ## Isotonic regression diff --git a/docs/mllib-linear-methods.md b/docs/mllib-linear-methods.md index 2879d88416..2d3ec4ca24 100644 --- a/docs/mllib-linear-methods.md +++ b/docs/mllib-linear-methods.md @@ -2,6 +2,21 @@ layout: global title: Linear Methods - RDD-based API displayTitle: Linear Methods - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-migration-guides.md b/docs/mllib-migration-guides.md index ea6f93fcf6..b746b96e19 100644 --- a/docs/mllib-migration-guides.md +++ b/docs/mllib-migration-guides.md @@ -2,6 +2,21 @@ layout: global title: Old Migration Guides - MLlib displayTitle: Old Migration Guides - MLlib +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The migration guide for the current Spark version is kept on the [MLlib Guide main page](ml-guide.html#migration-guide). diff --git a/docs/mllib-naive-bayes.md b/docs/mllib-naive-bayes.md index 7471d18a0d..09b15876a3 100644 --- a/docs/mllib-naive-bayes.md +++ b/docs/mllib-naive-bayes.md @@ -2,6 +2,21 @@ layout: global title: Naive Bayes - RDD-based API displayTitle: Naive Bayes - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- [Naive Bayes](http://en.wikipedia.org/wiki/Naive_Bayes_classifier) is a simple diff --git a/docs/mllib-optimization.md b/docs/mllib-optimization.md index 04758903da..f2e128ec21 100644 --- a/docs/mllib-optimization.md +++ b/docs/mllib-optimization.md @@ -2,6 +2,21 @@ layout: global title: Optimization - RDD-based API displayTitle: Optimization - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-pmml-model-export.md b/docs/mllib-pmml-model-export.md index f567565437..fbc14cefd1 100644 --- a/docs/mllib-pmml-model-export.md +++ b/docs/mllib-pmml-model-export.md @@ -2,6 +2,21 @@ layout: global title: PMML model export - RDD-based API displayTitle: PMML model export - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/mllib-statistics.md b/docs/mllib-statistics.md index 6bf013f3b0..4698d3e634 100644 --- a/docs/mllib-statistics.md +++ b/docs/mllib-statistics.md @@ -2,6 +2,21 @@ layout: global title: Basic Statistics - RDD-based API displayTitle: Basic Statistics - RDD-based API +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/monitoring.md b/docs/monitoring.md index bf77b4abd6..751f8c6c48 100644 --- a/docs/monitoring.md +++ b/docs/monitoring.md @@ -2,6 +2,21 @@ layout: global title: Monitoring and Instrumentation description: Monitoring, metrics, and instrumentation guide for Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- There are several ways to monitor Spark applications: web UIs, metrics, and external instrumentation. diff --git a/docs/programming-guide.md b/docs/programming-guide.md index f8b8f74c53..986bea3f71 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -2,6 +2,21 @@ layout: global title: Spark Programming Guide redirect: rdd-programming-guide.html +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- This document has moved [here](rdd-programming-guide.html). diff --git a/docs/quick-start.md b/docs/quick-start.md index 8bf0d6ee6b..93abb25f20 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -2,6 +2,21 @@ layout: global title: Quick Start description: Quick start tutorial for Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/rdd-programming-guide.md b/docs/rdd-programming-guide.md index 7f427659b1..b568e94c67 100644 --- a/docs/rdd-programming-guide.md +++ b/docs/rdd-programming-guide.md @@ -2,6 +2,21 @@ layout: global title: RDD Programming Guide description: Spark SPARK_VERSION_SHORT programming guide in Java, Scala and Python +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md index 7e26d77b05..f1000346cb 100644 --- a/docs/running-on-kubernetes.md +++ b/docs/running-on-kubernetes.md @@ -1,6 +1,21 @@ --- layout: global title: Running Spark on Kubernetes +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index a07773c1c7..907f414e5d 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -1,6 +1,21 @@ --- layout: global title: Running Spark on Mesos +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/running-on-yarn.md b/docs/running-on-yarn.md index 261f7e31ea..dc93e9cea5 100644 --- a/docs/running-on-yarn.md +++ b/docs/running-on-yarn.md @@ -1,6 +1,21 @@ --- layout: global title: Running Spark on YARN +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/security.md b/docs/security.md index 20492d871b..a4556e242c 100644 --- a/docs/security.md +++ b/docs/security.md @@ -2,6 +2,21 @@ layout: global displayTitle: Spark Security title: Security +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). {:toc} diff --git a/docs/spark-standalone.md b/docs/spark-standalone.md index ef99f05828..2ca3ee6aa7 100644 --- a/docs/spark-standalone.md +++ b/docs/spark-standalone.md @@ -1,6 +1,21 @@ --- layout: global title: Spark Standalone Mode +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/sparkr.md b/docs/sparkr.md index dbb6124100..26eeae6fe0 100644 --- a/docs/sparkr.md +++ b/docs/sparkr.md @@ -2,6 +2,21 @@ layout: global displayTitle: SparkR (R on Spark) title: SparkR (R on Spark) +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/sql-data-sources-avro.md b/docs/sql-data-sources-avro.md index 40d53fbe98..b73fd4f5c0 100644 --- a/docs/sql-data-sources-avro.md +++ b/docs/sql-data-sources-avro.md @@ -1,6 +1,21 @@ --- layout: global title: Apache Avro Data Source Guide +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/sql-data-sources-hive-tables.md b/docs/sql-data-sources-hive-tables.md index 14773ca4ec..0c51c11079 100644 --- a/docs/sql-data-sources-hive-tables.md +++ b/docs/sql-data-sources-hive-tables.md @@ -2,6 +2,21 @@ layout: global title: Hive Tables displayTitle: Hive Tables +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-data-sources-jdbc.md b/docs/sql-data-sources-jdbc.md index a2b14620be..3119ec004b 100644 --- a/docs/sql-data-sources-jdbc.md +++ b/docs/sql-data-sources-jdbc.md @@ -2,6 +2,21 @@ layout: global title: JDBC To Other Databases displayTitle: JDBC To Other Databases +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-data-sources-json.md b/docs/sql-data-sources-json.md index f84336b571..4ce4897189 100644 --- a/docs/sql-data-sources-json.md +++ b/docs/sql-data-sources-json.md @@ -2,6 +2,21 @@ layout: global title: JSON Files displayTitle: JSON Files +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. ---
diff --git a/docs/sql-data-sources-load-save-functions.md b/docs/sql-data-sources-load-save-functions.md index b45561cf93..a7efb9347a 100644 --- a/docs/sql-data-sources-load-save-functions.md +++ b/docs/sql-data-sources-load-save-functions.md @@ -2,6 +2,21 @@ layout: global title: Generic Load/Save Functions displayTitle: Generic Load/Save Functions +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-data-sources-orc.md b/docs/sql-data-sources-orc.md index ef07d2fa2e..45bff17c6c 100644 --- a/docs/sql-data-sources-orc.md +++ b/docs/sql-data-sources-orc.md @@ -2,6 +2,21 @@ layout: global title: ORC Files displayTitle: ORC Files +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Since Spark 2.3, Spark supports a vectorized ORC reader with a new ORC file format for ORC files. diff --git a/docs/sql-data-sources-parquet.md b/docs/sql-data-sources-parquet.md index f6e03fbaff..b5309870f4 100644 --- a/docs/sql-data-sources-parquet.md +++ b/docs/sql-data-sources-parquet.md @@ -2,6 +2,21 @@ layout: global title: Parquet Files displayTitle: Parquet Files +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-data-sources-troubleshooting.md b/docs/sql-data-sources-troubleshooting.md index 5775eb8b5c..993d9bf686 100644 --- a/docs/sql-data-sources-troubleshooting.md +++ b/docs/sql-data-sources-troubleshooting.md @@ -2,6 +2,21 @@ layout: global title: Troubleshooting displayTitle: Troubleshooting +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * The JDBC driver class must be visible to the primordial class loader on the client session and on all executors. This is because Java's DriverManager class does a security check that results in it ignoring all drivers not visible to the primordial class loader when one goes to open a connection. One convenient way to do this is to modify compute_classpath.sh on all worker nodes to include your driver JARs. diff --git a/docs/sql-data-sources.md b/docs/sql-data-sources.md index 636636af62..d908aac214 100644 --- a/docs/sql-data-sources.md +++ b/docs/sql-data-sources.md @@ -2,6 +2,21 @@ layout: global title: Data Sources displayTitle: Data Sources +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- diff --git a/docs/sql-distributed-sql-engine.md b/docs/sql-distributed-sql-engine.md index 66d6fdaf90..fc849d3912 100644 --- a/docs/sql-distributed-sql-engine.md +++ b/docs/sql-distributed-sql-engine.md @@ -2,6 +2,21 @@ layout: global title: Distributed SQL Engine displayTitle: Distributed SQL Engine +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-getting-started.md b/docs/sql-getting-started.md index 0c3f0fb206..5d18c48879 100644 --- a/docs/sql-getting-started.md +++ b/docs/sql-getting-started.md @@ -2,6 +2,21 @@ layout: global title: Getting Started displayTitle: Getting Started +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-keywords.md b/docs/sql-keywords.md index 40f75863e2..62378d9b40 100644 --- a/docs/sql-keywords.md +++ b/docs/sql-keywords.md @@ -2,6 +2,21 @@ layout: global title: Spark SQL Keywords displayTitle: Spark SQL Keywords +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- When `spark.sql.parser.ansi.enabled` is true, Spark SQL has two kinds of keywords: diff --git a/docs/sql-migration-guide-hive-compatibility.md b/docs/sql-migration-guide-hive-compatibility.md index dd7b062257..857e9748dd 100644 --- a/docs/sql-migration-guide-hive-compatibility.md +++ b/docs/sql-migration-guide-hive-compatibility.md @@ -2,6 +2,21 @@ layout: global title: Compatibility with Apache Hive displayTitle: Compatibility with Apache Hive +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-migration-guide-upgrade.md b/docs/sql-migration-guide-upgrade.md index 3ba89c0bb5..fef801275b 100644 --- a/docs/sql-migration-guide-upgrade.md +++ b/docs/sql-migration-guide-upgrade.md @@ -2,6 +2,21 @@ layout: global title: Spark SQL Upgrading Guide displayTitle: Spark SQL Upgrading Guide +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-migration-guide.md b/docs/sql-migration-guide.md index 71d83e8a55..4c23147106 100644 --- a/docs/sql-migration-guide.md +++ b/docs/sql-migration-guide.md @@ -2,6 +2,21 @@ layout: global title: Migration Guide displayTitle: Migration Guide +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * [Spark SQL Upgrading Guide](sql-migration-guide-upgrade.html) diff --git a/docs/sql-performance-tuning.md b/docs/sql-performance-tuning.md index 7c7c4a8155..68569744af 100644 --- a/docs/sql-performance-tuning.md +++ b/docs/sql-performance-tuning.md @@ -2,6 +2,21 @@ layout: global title: Performance Tuning displayTitle: Performance Tuning +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 9c85a15827..0a4d07ea37 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -2,6 +2,21 @@ layout: global displayTitle: Spark SQL, DataFrames and Datasets Guide title: Spark SQL and DataFrames +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Spark SQL is a Spark module for structured data processing. Unlike the basic Spark RDD API, the interfaces provided diff --git a/docs/sql-pyspark-pandas-with-arrow.md b/docs/sql-pyspark-pandas-with-arrow.md index d18ca0beb0..f40b33eb97 100644 --- a/docs/sql-pyspark-pandas-with-arrow.md +++ b/docs/sql-pyspark-pandas-with-arrow.md @@ -2,6 +2,21 @@ layout: global title: PySpark Usage Guide for Pandas with Apache Arrow displayTitle: PySpark Usage Guide for Pandas with Apache Arrow +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/sql-reference.md b/docs/sql-reference.md index 88d0596f38..ee99ed8f43 100644 --- a/docs/sql-reference.md +++ b/docs/sql-reference.md @@ -2,6 +2,21 @@ layout: global title: Reference displayTitle: Reference +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * Table of contents diff --git a/docs/storage-openstack-swift.md b/docs/storage-openstack-swift.md index 3264711df7..6bdcaa7ccb 100644 --- a/docs/storage-openstack-swift.md +++ b/docs/storage-openstack-swift.md @@ -1,6 +1,21 @@ --- layout: global title: Accessing OpenStack Swift from Spark +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Spark's support for Hadoop InputFormat allows it to process data in OpenStack Swift using the diff --git a/docs/streaming-custom-receivers.md b/docs/streaming-custom-receivers.md index a83ebd9449..a37b961243 100644 --- a/docs/streaming-custom-receivers.md +++ b/docs/streaming-custom-receivers.md @@ -1,6 +1,21 @@ --- layout: global title: Spark Streaming Custom Receivers +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Spark Streaming can receive streaming data from any arbitrary data source beyond diff --git a/docs/streaming-kafka-0-10-integration.md b/docs/streaming-kafka-0-10-integration.md index 3fb6271054..d8fd6724e9 100644 --- a/docs/streaming-kafka-0-10-integration.md +++ b/docs/streaming-kafka-0-10-integration.md @@ -1,6 +1,21 @@ --- layout: global title: Spark Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The Spark Streaming integration for Kafka 0.10 provides simple parallelism, 1:1 correspondence between Kafka diff --git a/docs/streaming-kafka-integration.md b/docs/streaming-kafka-integration.md index 0ec5a31ddf..6fa363285f 100644 --- a/docs/streaming-kafka-integration.md +++ b/docs/streaming-kafka-integration.md @@ -1,6 +1,21 @@ --- layout: global title: Spark Streaming + Kafka Integration Guide +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- [Apache Kafka](https://kafka.apache.org/) is publish-subscribe messaging rethought as a distributed, partitioned, diff --git a/docs/streaming-kinesis-integration.md b/docs/streaming-kinesis-integration.md index 4a1812bbb4..fd6d776045 100644 --- a/docs/streaming-kinesis-integration.md +++ b/docs/streaming-kinesis-integration.md @@ -1,6 +1,21 @@ --- layout: global title: Spark Streaming + Kinesis Integration +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- [Amazon Kinesis](http://aws.amazon.com/kinesis/) is a fully managed service for real-time processing of streaming data at massive scale. The Kinesis receiver creates an input DStream using the Kinesis Client Library (KCL) provided by Amazon under the Amazon Software License (ASL). diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md index 30a0160a65..854beb60f1 100644 --- a/docs/streaming-programming-guide.md +++ b/docs/streaming-programming-guide.md @@ -3,6 +3,21 @@ layout: global displayTitle: Spark Streaming Programming Guide title: Spark Streaming description: Spark Streaming programming guide and tutorial for Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/structured-streaming-kafka-integration.md b/docs/structured-streaming-kafka-integration.md index 7c043acb49..9e1bbc06e5 100644 --- a/docs/structured-streaming-kafka-integration.md +++ b/docs/structured-streaming-kafka-integration.md @@ -1,6 +1,21 @@ --- layout: global title: Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- Structured Streaming integration for Kafka 0.10 to read data from and write data to Kafka. diff --git a/docs/structured-streaming-programming-guide.md b/docs/structured-streaming-programming-guide.md index d425bcc125..2b6940d055 100644 --- a/docs/structured-streaming-programming-guide.md +++ b/docs/structured-streaming-programming-guide.md @@ -2,6 +2,21 @@ layout: global displayTitle: Structured Streaming Programming Guide title: Structured Streaming Programming Guide +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped). diff --git a/docs/submitting-applications.md b/docs/submitting-applications.md index d6b663ea8d..cbc6f86919 100644 --- a/docs/submitting-applications.md +++ b/docs/submitting-applications.md @@ -1,6 +1,21 @@ --- layout: global title: Submitting Applications +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- The `spark-submit` script in Spark's `bin` directory is used to launch applications on a cluster. diff --git a/docs/tuning.md b/docs/tuning.md index 43acacb98c..222f8720ce 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -3,6 +3,21 @@ layout: global displayTitle: Tuning Spark title: Tuning description: Tuning and performance optimization guide for Spark SPARK_VERSION_SHORT +license: | + Licensed to the Apache Software Foundation (ASF) under one or more + contributor license agreements. See the NOTICE file distributed with + this work for additional information regarding copyright ownership. + The ASF licenses this file to You under the Apache License, Version 2.0 + (the "License"); you may not use this file except in compliance with + the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. --- * This will become a table of contents (this text will be scraped).