[MINOR][SQL] Locality does not need to be implemented
## What changes were proposed in this pull request? `HadoopFileWholeTextReader` and `HadoopFileLinesReader` will be eventually called in `FileSourceScanExec`. In fact, locality has been implemented in `FileScanRDD`, even if we implement it in `HadoopFileWholeTextReader ` and `HadoopFileLinesReader`, it would be useless. So I think these `TODO` can be removed. ## How was this patch tested? N/A Closes #23339 from 10110346/noneededtodo. Authored-by: liuxian <liu.xian3@zte.com.cn> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This commit is contained in:
parent
aa0d4ca8ba
commit
98ecda3e8e
|
@ -51,7 +51,7 @@ class HadoopFileLinesReader(
|
|||
new Path(new URI(file.filePath)),
|
||||
file.start,
|
||||
file.length,
|
||||
// TODO: Implement Locality
|
||||
// The locality is decided by `getPreferredLocations` in `FileScanRDD`.
|
||||
Array.empty)
|
||||
val attemptId = new TaskAttemptID(new TaskID(new JobID(), TaskType.MAP, 0), 0)
|
||||
val hadoopAttemptContext = new TaskAttemptContextImpl(conf, attemptId)
|
||||
|
|
|
@ -40,7 +40,7 @@ class HadoopFileWholeTextReader(file: PartitionedFile, conf: Configuration)
|
|||
Array(new Path(new URI(file.filePath))),
|
||||
Array(file.start),
|
||||
Array(file.length),
|
||||
// TODO: Implement Locality
|
||||
// The locality is decided by `getPreferredLocations` in `FileScanRDD`.
|
||||
Array.empty[String])
|
||||
val attemptId = new TaskAttemptID(new TaskID(new JobID(), TaskType.MAP, 0), 0)
|
||||
val hadoopAttemptContext = new TaskAttemptContextImpl(conf, attemptId)
|
||||
|
|
Loading…
Reference in a new issue