[SPARK-5789][SQL]Throw a better error message if JsonRDD.parseJson encounters unrecoverable parsing errors.
Author: Yin Huai <yhuai@databricks.com> Closes #4582 from yhuai/jsonErrorMessage and squashes the following commits: 152dbd4 [Yin Huai] Update error message. 1466256 [Yin Huai] Throw a better error message when a JSON object in the input dataset span multiple records (lines for files or strings for an RDD of strings).
This commit is contained in:
parent
2cbb3e433a
commit
2e0c084528
|
@ -306,6 +306,10 @@ private[sql] object JsonRDD extends Logging {
|
|||
val parsed = mapper.readValue(record, classOf[Object]) match {
|
||||
case map: java.util.Map[_, _] => scalafy(map).asInstanceOf[Map[String, Any]] :: Nil
|
||||
case list: java.util.List[_] => scalafy(list).asInstanceOf[Seq[Map[String, Any]]]
|
||||
case _ =>
|
||||
sys.error(
|
||||
s"Failed to parse record $record. Please make sure that each line of the file " +
|
||||
"(or each string in the RDD) is a valid JSON object or an array of JSON objects.")
|
||||
}
|
||||
|
||||
parsed
|
||||
|
|
Loading…
Reference in a new issue