ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Remove non-obvious conf settings from TPCDS benchmark

Browse source 
## What changes were proposed in this pull request?

My fault -- these 2 conf entries are mysteriously hidden inside the benchmark code and makes it non-obvious to disable whole stage codegen and/or the vectorized parquet reader.

PS: Didn't attach a JIRA as this change should otherwise be a no-op (both these conf are enabled by default in Spark)

## How was this patch tested?

N/A

Author: Sameer Agarwal <sameer@databricks.com>

Closes #13726 from sameeragarwal/tpcds-conf.
This commit is contained in:
Sameer Agarwal 2016-06-17 09:47:41 -07:00 committed by Herman van Hovell
parent ef43b4ed87
commit 34d6c4cd11
1 changed files with 0 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/TPCDSQueryBenchmark.scala
View file

@ -64,8 +64,6 @@ object TPCDSQueryBenchmark {
require(dataLocation.nonEmpty, require(dataLocation.nonEmpty,
"please modify the value of dataLocation to point to your local TPCDS data") "please modify the value of dataLocation to point to your local TPCDS data")
val tableSizes = setupTables(dataLocation) val tableSizes = setupTables(dataLocation)
spark.conf.set(SQLConf.PARQUET_VECTORIZED_READER_ENABLED.key, "true")
spark.conf.set(SQLConf.WHOLESTAGE_CODEGEN_ENABLED.key, "true")
queries.foreach { name => queries.foreach { name =>
val queryString = fileToString(new File(Thread.currentThread().getContextClassLoader val queryString = fileToString(new File(Thread.currentThread().getContextClassLoader
.getResource(s"tpcds/$name.sql").getFile)) .getResource(s"tpcds/$name.sql").getFile))