[SPARK-34786][SQL][FOLLOWUP] Explicitly declare DecimalType(20, 0) for Parquet UINT_64
### What changes were proposed in this pull request? Explicitly declare DecimalType(20, 0) for Parquet UINT_64, avoid use DecimalType.LongDecimal which only happens to have 20 as precision. https://github.com/apache/spark/pull/31960#discussion_r622691560 ### Why are the changes needed? fix ambiguity ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? not needed, just current CI pass Closes #32390 from yaooqinn/SPARK-34786-F. Authored-by: Kent Yao <yao@apache.org> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This commit is contained in:
parent
74b93261af
commit
771356555c
|
@ -141,7 +141,9 @@ class ParquetToSparkSchemaConverter(
|
|||
originalType match {
|
||||
case INT_64 | null => LongType
|
||||
case DECIMAL => makeDecimalType(Decimal.MAX_LONG_DIGITS)
|
||||
case UINT_64 => DecimalType.LongDecimal
|
||||
// The precision to hold the largest unsigned long is:
|
||||
// `java.lang.Long.toUnsignedString(-1).length` = 20
|
||||
case UINT_64 => DecimalType(20, 0)
|
||||
case TIMESTAMP_MICROS => TimestampType
|
||||
case TIMESTAMP_MILLIS => TimestampType
|
||||
case _ => illegalType()
|
||||
|
|
Loading…
Reference in a new issue