spark-instrumented-optimizer/sql/core
Kousuke Saruta 6cd0bef7fe
[SPARK-31416][SQL] Check more strictly that a field name can be used as a valid Java identifier for codegen
### What changes were proposed in this pull request?

Check more strictly that a field name can be used as a valid Java identifier in `ScalaReflection.serializerFor`
To check that, `SourceVersion` is used so that we need not add reserved keywords to be checked manually for the future Java versions (e.g, underscore, var, yield), .

### Why are the changes needed?

In the current implementation, `enum` is not checked even though it's a reserved keyword.
Also, there are lots of characters and sequences of character including numeric literals but they are not checked.
So we can't get better error message with following code.
```
case class  Data(`0`: Int)
Seq(Data(1)).toDF.show

20/04/11 03:24:24 ERROR CodeGenerator: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 43, Column 1: Expression "value_0 = value_3" is not a type
org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 43, Column 1: Expression "value_0 = value_3" is not a type

...

```

### Does this PR introduce any user-facing change?

Yes. With this change and the code example above, we can get following error message.
```
java.lang.UnsupportedOperationException: `0` is not a valid identifier of Java and cannot be used as field name
- root class: "Data"

...
```

### How was this patch tested?

Add another assertion to existing test case.

Closes #28184 from sarutak/improve-identifier-check.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
2020-04-12 13:14:41 -07:00
..
benchmarks [SPARK-31359][SQL] Speed up timestamps rebasing 2020-04-09 05:23:52 +00:00
src [SPARK-31416][SQL] Check more strictly that a field name can be used as a valid Java identifier for codegen 2020-04-12 13:14:41 -07:00
v1.2/src [SPARK-31359][SQL] Speed up timestamps rebasing 2020-04-09 05:23:52 +00:00
v2.3/src [SPARK-31359][SQL] Speed up timestamps rebasing 2020-04-09 05:23:52 +00:00
pom.xml [SPARK-31021][SQL] Support MariaDB Kerberos login in JDBC connector 2020-04-09 09:20:02 -07:00