74d2f04183
## What changes were proposed in this pull request? This PR aims to improve `printSchema` to be able to print up to the given level of the schema. ```scala scala> val df = Seq((1,(2,(3,4)))).toDF df: org.apache.spark.sql.DataFrame = [_1: int, _2: struct<_1: int, _2: struct<_1: int, _2: int>>] scala> df.printSchema root |-- _1: integer (nullable = false) |-- _2: struct (nullable = true) | |-- _1: integer (nullable = false) | |-- _2: struct (nullable = true) | | |-- _1: integer (nullable = false) | | |-- _2: integer (nullable = false) scala> df.printSchema(1) root |-- _1: integer (nullable = false) |-- _2: struct (nullable = true) scala> df.printSchema(2) root |-- _1: integer (nullable = false) |-- _2: struct (nullable = true) | |-- _1: integer (nullable = false) | |-- _2: struct (nullable = true) scala> df.printSchema(3) root |-- _1: integer (nullable = false) |-- _2: struct (nullable = true) | |-- _1: integer (nullable = false) | |-- _2: struct (nullable = true) | | |-- _1: integer (nullable = false) | | |-- _2: integer (nullable = false) ``` ## How was this patch tested? Pass the Jenkins with the newly added test case. Closes #24098 from dongjoon-hyun/SPARK-27166. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |