2020-07-27 04:49:21 -04:00
|
|
|
.. Licensed to the Apache Software Foundation (ASF) under one
|
|
|
|
or more contributor license agreements. See the NOTICE file
|
|
|
|
distributed with this work for additional information
|
|
|
|
regarding copyright ownership. The ASF licenses this file
|
|
|
|
to you under the Apache License, Version 2.0 (the
|
|
|
|
"License"); you may not use this file except in compliance
|
|
|
|
with the License. You may obtain a copy of the License at
|
|
|
|
|
|
|
|
.. http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
|
|
|
|
.. Unless required by applicable law or agreed to in writing,
|
|
|
|
software distributed under the License is distributed on an
|
|
|
|
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
|
|
KIND, either express or implied. See the License for the
|
|
|
|
specific language governing permissions and limitations
|
|
|
|
under the License.
|
|
|
|
|
|
|
|
|
|
|
|
===============
|
|
|
|
Migration Guide
|
|
|
|
===============
|
|
|
|
|
2020-09-01 02:08:03 -04:00
|
|
|
This page describes the migration guide specific to PySpark.
|
2020-08-10 02:41:32 -04:00
|
|
|
|
|
|
|
.. toctree::
|
|
|
|
:maxdepth: 2
|
|
|
|
|
2021-05-03 02:34:24 -04:00
|
|
|
pyspark_3.1_to_3.2
|
2020-08-10 02:41:32 -04:00
|
|
|
pyspark_2.4_to_3.0
|
|
|
|
pyspark_2.3_to_2.4
|
|
|
|
pyspark_2.3.0_to_2.3.1_above
|
|
|
|
pyspark_2.2_to_2.3
|
|
|
|
pyspark_1.4_to_1.5
|
|
|
|
pyspark_1.0_1.2_to_1.3
|
2021-01-07 19:28:31 -05:00
|
|
|
|
|
|
|
|
|
|
|
Many items of other migration guides can also be applied when migrating PySpark to higher versions because PySpark internally shares other components.
|
|
|
|
Please also refer other migration guides:
|
|
|
|
|
2021-02-22 21:18:47 -05:00
|
|
|
- `Migration Guide: Spark Core <https://spark.apache.org/docs/latest/core-migration-guide.html>`_
|
|
|
|
- `Migration Guide: SQL, Datasets and DataFrame <https://spark.apache.org/docs/latest/sql-migration-guide.html>`_
|
|
|
|
- `Migration Guide: Structured Streaming <https://spark.apache.org/docs/latest/ss-migration-guide.html>`_
|
|
|
|
- `Migration Guide: MLlib (Machine Learning) <https://spark.apache.org/docs/latest/ml-migration-guide.html>`_
|
2021-01-07 19:28:31 -05:00
|
|
|
|