69 lines
2.5 KiB
Markdown
69 lines
2.5 KiB
Markdown
|
|
# Big Todos for CIDR:
|
|
|
|
- [ ] Clone the main [Spark Optimizer Rules](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala#L72)
|
|
- [x] PushProjectionThroughUnion,
|
|
- [ ] PushProjectionThroughLimit,
|
|
- [x] ReorderJoin,
|
|
- [ ] EliminateOuterJoin,
|
|
- [ ] PushDownPredicates,
|
|
- [ ] PushDownLeftSemiAntiJoin,
|
|
- [ ] PushLeftSemiLeftAntiThroughJoin,
|
|
- [ ] LimitPushDown,
|
|
- [ ] LimitPushDownThroughWindow,
|
|
- [ ] ColumnPruning,
|
|
- [ ] GenerateOptimization,
|
|
- [ ] CollapseRepartition,
|
|
- [ ] CollapseProject,
|
|
- [ ] OptimizeWindowFunctions,
|
|
- [ ] CollapseWindow,
|
|
- [ ] EliminateOffsets,
|
|
- [ ] EliminateLimits,
|
|
- [ ] CombineUnions,
|
|
- [ ] OptimizeRepartition,
|
|
- [ ] TransposeWindow,
|
|
- [ ] NullPropagation,
|
|
- [ ] NullDownPropagation,
|
|
- [ ] ConstantPropagation,
|
|
- [ ] FoldablePropagation,
|
|
- [ ] OptimizeIn,
|
|
- [ ] OptimizeRand,
|
|
- [ ] ConstantFolding,
|
|
- [ ] EliminateAggregateFilter,
|
|
- [ ] ReorderAssociativeOperator,
|
|
- [ ] LikeSimplification,
|
|
- [ ] BooleanSimplification,
|
|
- [ ] SimplifyConditionals,
|
|
- [ ] PushFoldableIntoBranches,
|
|
- [ ] RemoveDispensableExpressions,
|
|
- [ ] SimplifyBinaryComparison,
|
|
- [ ] ReplaceNullWithFalseInPredicate,
|
|
- [ ] PruneFilters,
|
|
- [ ] SimplifyCasts,
|
|
- [ ] SimplifyCaseConversionExpressions,
|
|
- [ ] RewriteCorrelatedScalarSubquery,
|
|
- [ ] RewriteLateralSubquery,
|
|
- [ ] EliminateSerialization,
|
|
- [ ] RemoveRedundantAliases,
|
|
- [ ] RemoveRedundantAggregates,
|
|
- [ ] UnwrapCastInBinaryComparison,
|
|
- [ ] RemoveNoopOperators,
|
|
- [ ] OptimizeUpdateFields,
|
|
- [ ] SimplifyExtractValueOps,
|
|
- [ ] OptimizeCsvJsonExprs,
|
|
- [ ] CombineConcats,
|
|
- [ ] PushdownPredicatesAndPruneColumnsForCTEDef
|
|
- [ ] Codegen: Output a Spark-like optimizer based on the above rules
|
|
- [x] Logic generation
|
|
- [ ] Compilation pipeline to streamline testing
|
|
- [ ] Test: Do these rules generate the same results as spark?
|
|
- [ ] Apply the rule merge optimization
|
|
- [ ] Generate a merged decision tree
|
|
- [ ] Codegen: Generate a rule for the joint BDD
|
|
- [ ] Test: Do these rules generate the same results as spark?
|
|
- [ ] Test: Can the generated rule be plugged back into spark?
|
|
- [ ] Apply the view maintenance optimization
|
|
- [ ] Recursive Pattern-Matching Logic
|
|
- [ ] Codegen: Generate a new optimizer
|
|
- [ ] Test: Do these rules generate the same results as spark?
|