ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-36312][SQL][FOLLOWUP] Add back ParquetSchemaConverter.checkFieldNames

Browse source 
### What changes were proposed in this pull request?
Add back ParquetSchemaConverter.checkFieldNames()

### Why are the changes needed?
Fix code

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?

Closes #33552 from AngersZhuuuu/SPARK-36312-FOLLOWUP.

Authored-by: Angerszhuuuu <angers.zhu@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This commit is contained in:
Angerszhuuuu 2021-07-28 14:38:23 +08:00 committed by Wenchen Fan
parent 86f44578e5
commit f086c17b8e
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala
View file

@ -591,6 +591,16 @@ private[sql] object ParquetSchemaConverter {
}
}
def checkFieldNames(schema: StructType): Unit = {
schema.foreach { field =>
checkFieldName(field.name)
field.dataType match {
case s: StructType => checkFieldNames(s)
case _ =>
}
}
}
def checkConversionRequirement(f: => Boolean, message: String): Unit = {
if (!f) {
throw new AnalysisException(message)